Django default input validation accepts special caracters

[1337 Shadow Hacker]

Currently, when you order a security audit on a Django project from any of the firms I've seen so far (including my own), all inputs fall short on stuff like:

"First name input: allows special caracters such as <>/"' which may cause a security issue with further developments done on the same database but outside Django".

As far as I can imagine, special caracters would be acceptable on inputs that should accept code or some kind of math, which is not the case for most inputs.

Django should harden default input validation to make it easier for Django projects to get a good grade on security audits, without having to go over all fields to setup basic input validators.

[Kacper Szmigiel]

Hello!

Maybe some `special_characters` bool field on models.CharField with default to `False` would do the job?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/EiNHz_fmHLmQXZ5ChTG0qrnp8BrP0s75szk9oDolStpIyMSz71B3yesI7U6K8QZNkUmeAN6v6zMhExwhwcbtGNeaOUgubDOIDK-Q4cVFvOw%3D%40protonmail.com.

[Adam Johnson]

I am against adding validation here. See the classic Falsehoods Programmers Believe About Names: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ .

Here are some characters that have caused security issues in the past (e.g. URL bar display), but I'd reckon are legitimate in some human names:

• non-breaking space
• zero-width space
  
• RTL or LTR markers
• Quote marks
  

If you do want to block this, we don't need a new argument to Charfield. You can implement a custom validator, and even pair it with a database constraint. But you might block legitimate users.

The only change we should be making is moving from separate first_name + last_name fields to solely a name field, since *many* people don't fit into that. I think there's a ticket, but there are massive backwards compatibility concerns.

P.S. never forget the case of Mr Null: https://www.wired.com/2015/11/null/

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFfZ%2Bb4GNSksSUm%3De8bQAp1t1qgDkLvNpitYjLmY1QHb94vs7w%40mail.gmail.com.

--

Adam

[1337 Shadow Hacker]

Well, at least in my country there's a law that tells what characters are allowed in names, anyway, a single name field would be cool but off topic here: "first name" was used here as an example to illustrate that Django projects are audited as insecure because there is no input validation at all by default.

For example, django.core.validators does supply a number of useful validators, but perhaps it could provide more ie. to restrict special caracters, raise warnings for fields which haven't any validators at all and force the user to set validators=[] like ForeignKey currently does for on_delete.

Not sure if anything can/should be done though, ideas welcome :)

[Matthew Pava]

Thank you so much for sharing that, Adam. I’ve always wondered what the best way was to deal with names. The systems I work with should be able to handle names from all countries of the world. I find you kalzumeus link only slightly helpful, though. It explains what we shouldn’t do. I’d rather just have an explanation of what we should do to handle names. Seriously, should we have just one field that a person can enter their full name? Or multiple fields?

 

And perhaps Django should lead on this. As bad as the backwards compatibility concerns are, we should probably just address it once and for all.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM2CsjYg-8DJ2AyOpbbnn-vR-CUK9DSSf-1_wywJejg4aQ%40mail.gmail.com.

[Adam Johnson]

Here's a reference from W3C on "what you should do": https://www.w3.org/International/questions/qa-personal-names#singlefield - with a great intro.

I'm not sure what a django.contrib.auth migration path could look like. We'd probably want to preserve the first_name + last_name behaviour as an option as combining without breaking systems will be very difficult.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/697a2ac6964d4a94b2fe96e1331e36f7%40iss2.ISS.LOCAL.

--

Adam

[Curtis Maloney]

Just my 0.02 $CURRENCY...

Interesting they're limiting the input for a security issue [at least from the example] that manifest from not escaping _output_.

Seems related to something I learned early in my web career... about not storing values escaped, because you don't know which medium it needs escaping for -- e.g. HTML needs different escaping than URLs.

--

Curtis

[אורי‎]

On Tue, Aug 18, 2020 at 12:26 PM Adam Johnson <m...@adamj.eu> wrote:

The only change we should be making is moving from separate first_name + last_name fields to solely a name field, since *many* people don't fit into that. I think there's a ticket, but there are massive backwards compatibility concerns.

This may be true - not all people have first_name & last_name or want to use them online. But it's also convenient to be able to call a person by their first name, and also allow them to use their full name on the website. I checked popular websites - Facebook and Google require first name and last name, Twitter, GitHub and Stack Overflow require only a "name", and I heard that in some IP addresses and languages Facebook doesn't require a last name, but I tried and I couldn't delete my last name from Facebook. I have a formal last name, but I want to be registered only with my first name on Facebook, which I can't. And by the way, Facebook, Twitter and Github let me change my username, Google didn't. So I have to use a username which doesn't reflect my real name on Google (I changed my last name and the username reflects my former last name). The only way to use a different username on Google is to create a new account. But also if I change my username, my email address would change, and messages sent to my previous email address might not reach me.

אורי.

[Florian Apolloner]

On Wednesday, August 19, 2020 at 2:01:51 AM UTC+2 cur...@tinbrain.net wrote:

Seems related to something I learned early in my web career... about not storing values escaped, because you don't know which medium it needs escaping for -- e.g. HTML needs different escaping than URLs.

Exactly, this is the way to go. I'd switch any security firm that complains about "allowing special characters" in security audits in a heartbeat.

[1337 Shadow Hacker]

Thanks for the comment Florian, it's just basic hygiene really, don't leave open ports you don't need, never trust user inputs for characters they don't need, and so on.

[1337 Shadow Hacker]

And I'm sorry if I offended Mister <script type="text/javascript">alert("pwnd")</script> :)

[1337 Shadow Hacker]

Input validation is performed to ensure only properly formed data is entering the workflow in an information system, preventing malformed data from persisting in the database and triggering malfunction of various downstream components. Input validation should happen as early as possible in the data flow, preferably as soon as the data is received from the external party.

Data from all potentially untrusted sources should be subject to input validation, including not only Internet-facing web clients but also backend feeds over extranets, from suppliers, partners, vendors or regulators, each of which may be compromised on their own and start sending malformed data.

Input Validation should not be used as the primary method of preventing XSS, SQL Injection and other attacks which are covered in respective cheat sheets but can significantly contribute to reducing their impact if implemented properly.

https://cheatsheetseries.owasp.org/cheatsheets/Input_Validation_Cheat_Sheet.html

[1337 Shadow Hacker]

Opened an issue on the OWASP project, reporting the reasoning of the consensus made on this mailing list as best as I could:

https://github.com/OWASP/CheatSheetSeries/issues/472

Please feel free to comment or request changes on the issue.

[1337 Shadow Hacker]

> This may be true - not all people have first_name & last_name or want to use them online. But it's also convenient to be able to call a person by their first name, and also allow them to use their full name on the website.

I completely agree with you, for example on dating sites like speedy or things like that.

But when you are making a governmental website for example: you need actual identity.

Same goes when you're manipulating money: most countries require you to partake in the fight against money laundering and they will not allow you to take/pay people without a valid identity (KYC/KYB).

If you want to access a governmental or banking service online with a fake identity or forged documents you might just end up in jail.

[René Fleschenberg]

Hi,

> But when you are making a governmental website for example: you need
> actual identity.

Django makes it possible (I'd even say easy) to setup your forms /
models to enforce stricter requirements if you have to. We don't have to
treat these cases as the default. Backwards compatibility is the bigger
concern here.

Regards,
René

[1337 Shadow Hacker]

> Backwards compatibility is the bigger concern here.

I understand that, I believe there is always a way, because prior to 2008 when I switched to Django, I was commiter on a PHP library group that not only had the best code quality: but COMMITS to maintaining BC. Code that I have made prior to 2008 still works today with the last releases. Since then, I've been upgrading a lot of code, maintaining django packages with millions of downloads from Python 2 and Django 1 to nowadays and contribute the same to many libraries as much as I can, often proud to be the first to offer a patch, as such, I most definitely understand the cost of not maintaining BC.

I love BC, I think maintaining it is a challenge that always deserves as much time and effort as possible.

But, since Django broke BC for on_delete, which I accept to be "for the greater good", I just wanted to at least discuss how this could be done to improve the results of security audits for all Django users, and not just me and my 20 yrs of pro xp, I'm too 1337 to need it for myself, I thought it was worth discussing about it for others.

On a more emotional note, which I hope you will understand as **I've just been fired by one of the developers that I have admired most for more than a decade now, just for following OWASP recommandations**, no matter that I'm mentionning that my competitors do at least the same:

THANK YOU Rene,

For not completely discarding everything I offer for debate, it's been a while since django-developers make me feel like "just a craze", despite that I do actually maintain both governmental and financial websites in production which I develop and manage tech teams of, as part of different customers, and a lot of the money I make I actually throw away to other developers on R&D and other cool stuff which I wish I could be working on full time.

To others: keep up the great work, Django is a fantastic project and I love it, last year I took a few months to assess all other more recent languages and frameworks I could find, with the purpose of deciding what my stack would be for the next decade and guess what: it's still Django ;)

<3