Django formset hidden id field

[Cheng Guo]

Hello,

I have a formset and when I render it, Django would include this line in the HTML:

<input id="id_form-0-id" name="form-0-id" type="hidden">

I am curious what is the purpose of having an id field here. 

I mean in what situation would you use it. I did look through Django's documentation on formsetbut cannot find much documentation on this.

One answer I got is that this id field is the value of the primary key of the model bound to this form. It is there so that when the formset updates, people can use it to retrieve the corresponding record from the database.

Is the above explaination correct?

If this explaination is correct, then my next question is, wouldn't it be dangerous to expose the primary key like that? I can make a post call to your server with a modified pk which can mess up your database.

Thank you!

[Matthias Müller]

Without looking at the link I guess that you explantion is more or less correct. 

But it's not a security issue that the database is updated by a form. It has to be updated by a form. To make it a correct django form there is a hidden field with the CSRF token. This protects the database being updated from any illegal source. 

In your example there is this csrf missing, Most probably for didactical reasons. 

Refer to https://docs.djangoproject.com/en/1.8/ref/csrf/

Cheers

Matthias

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/18e0d250-c4a9-4060-ae4f-19afb57566e0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Cheng Guo]

Thank you! Yes, I forgot about the csrf. You are right, it would be difficult to fake the CSRF string.

Just in general, is it a good idea to expose primary keys like this? sometimes you can see them in urls too, like: www.yoursite/blog/1/,  1 would be the primary key of a blog.

[Matthias Müller]

Just in general, is it a good idea to expose primary keys like this? sometimes you can see them in urls too, like: www.yoursite/blog/1/,  1 would be the primary key of a blog.

It's an easy way to refer to an object. Unless there is a secure connection it's this is IMHO the best way to refer to the object. 

Of cause you can do it complicated ( with look up tables on the server etc. )  but the result matters.

And I like to keep my life and my apps simple 

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/285ee494-8b28-42cd-8af9-4cb33983a82c%40googlegroups.com.

[Luis Zárate]

Mmm I am not sure of this but I guest that this number is not a primary key (pk start in 1 not in 0 in  postgres and mysql), it is a formset control number used by formset for group fields in the server side ( for create forms in correct order also)

El miércoles, 27 de mayo de 2015, Matthias Müller <mmbl...@gmail.com> escribió:
>> Just in general, is it a good idea to expose primary keys like this? sometimes you can see them in urls too, like: www.yoursite/blog/1/,  1 would be the primary key of a blog.
>
> It's an easy way to refer to an object. Unless there is a secure connection it's this is IMHO the best way to refer to the object. 
> Of cause you can do it complicated ( with look up tables on the server etc. )  but the result matters.

> And I like to keep my life and my apps simple <https://mail.google.com/mail/e/softbank_ne_jp/337>

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAA2xsHSSy03DSHaBfT4RNxv4zYJUBTdySLvg__mjMDheSemE4w%40mail.gmail.com.

> For more options, visit https://groups.google.com/d/optout.
>

--

"La utopía sirve para caminar" Fernando Birri

[Peter of the Norse]

So what?  It’s quite likely that whoever is editing this row of the database, also has permissions to edit the other rows as well.  There’s no reason for them to go through the hassle of manually editing a hidden field when they can go to a different page and edit it there.

In general, primary keys are not security flaws.  While it’s a good idea to hide them from front-end pages, that’s mostly because they make URLs hard to read.  I have heard that you don’t want to use them publicly, because your competitors can use them to gauge your success, but that’s the kind of “Nice Problem to Have” that can wait until you’re bigger.

Thank you!

Peter of the Norse

Rahm...@Radio1190.org

[Carl Meyer]

On 06/28/2015 03:00 PM, Peter of the Norse wrote:
> On May 27, 2015, at 7:47 AM, Cheng Guo <chen...@gmail.com

> <mailto:chen...@gmail.com>> wrote:
>>
>> Hello,
>>
>> I have a formset and when I render it, Django would include this line
>> in the HTML:
>>
>> |<input id="id_form-0-id" name="form-0-id" type="hidden">|
>>
>> I am curious what is the purpose of having an id field here.
>>
>> I mean in what situation would you use it. I did look through
>> Django's documentation on formset

>> <https://docs.djangoproject.com/en/1.8/topics/forms/formsets/#can-delete>but

>> cannot find much documentation on this.
>>
>> One answer I got is that this id field is the value of the primary key
>> of the model bound to this form. It is there so that when the formset
>> updates, people can use it to retrieve the corresponding record from
>> the database.
>>
>> Is the above explaination correct?
>>
>> If this explaination is correct, then my next question is, wouldn't it
>> be dangerous to expose the primary key like that? I can make a post
>> call to your server with a modified pk which can mess up your database.
>
> So what? It’s quite likely that whoever is editing this row of the
> database, also has permissions to edit the other rows as well. There’s
> no reason for them to go through the hassle of manually editing a hidden
> field when they can go to a different page and edit it there.

That's a bad answer. It's common in many systems for a user to have
access to edit some records in a table but not others (this is often
known as "object-level permissions"). If it was really possible to edit
any row in the table by just manually editing the PK hidden field, that
would be a serious security flaw in formsets.

But it's not. Whatever queryset you pass to the model formset limits the
available rows for editing. The end user can edit the PK to refer to any
item in that queryset, but not any item in the table.

> In general, primary keys are not security flaws. While it’s a good idea
> to hide them from front-end pages, that’s mostly because they make URLs
> hard to read. I have heard that you don’t want to use them publicly,
> because your competitors can use them to gauge your success, but that’s
> the kind of “Nice Problem to Have” that can wait until you’re bigger.

Exposing primary keys themselves (e.g. in URLs) is not necessarily a
security flaw. Exposing a primary key in a hidden field that can be
edited to change a form to edit any row in a table often would be a
serious security flaw.


Carl

[Peter of the Norse]

> On Jul 2, 2015, at 7:46 PM, Carl Meyer <ca...@oddbird.net> wrote:
>
>> So what? It’s quite likely that whoever is editing this row of the
>> database, also has permissions to edit the other rows as well. There’s
>> no reason for them to go through the hassle of manually editing a hidden
>> field when they can go to a different page and edit it there.
>
> That's a bad answer. It's common in many systems for a user to have
> access to edit some records in a table but not others (this is often
> known as "object-level permissions"). If it was really possible to edit
> any row in the table by just manually editing the PK hidden field, that
> would be a serious security flaw in formsets.
>
> But it's not. Whatever queryset you pass to the model formset limits the
> available rows for editing. The end user can edit the PK to refer to any
> item in that queryset, but not any item in the table.
>
>> In general, primary keys are not security flaws. While it’s a good idea
>> to hide them from front-end pages, that’s mostly because they make URLs
>> hard to read. I have heard that you don’t want to use them publicly,
>> because your competitors can use them to gauge your success, but that’s
>> the kind of “Nice Problem to Have” that can wait until you’re bigger.
>
> Exposing primary keys themselves (e.g. in URLs) is not necessarily a
> security flaw. Exposing a primary key in a hidden field that can be
> edited to change a form to edit any row in a table often would be a
> serious security flaw.

You can’t have it both ways. Either exposing the PK is a security flaw or it isn’t. It’s just as easy for nefarious n’er-do-wells to edit the form’s URL as a hidden field. In either case, if you are using object-level permissions, more checks (not made automatic in Django) are necessary. Having the ID passed as a parameter doesn’t necessitate, hinder, or alleviate running these checks.

If you can come up with a method that associates form data with a database row that doesn’t involve the PK, I would love to know. Keep in mind that most tables only have the one unique identifier.

[Javier Guerra Giraldez]

On Sat, Jul 4, 2015 at 1:42 AM, Peter of the Norse
<Rahm...@radio1190.org> wrote:
> You can’t have it both ways. Either exposing the PK is a security flaw or it isn’t. It’s just as easy for nefarious n’er-do-wells to edit the form’s URL as a hidden field. In either case, if you are using object-level permissions, more checks (not made automatic in Django) are necessary. Having the ID passed as a parameter doesn’t necessitate, hinder, or alleviate running these checks.

the easiest way to make forget PKs a non-issue is to never do things like:

myobj = MyModel.objects.get(pk=pk)

instead, do things like:

myobj = get_object_or_404(request.user.accessible_obj_set, pk=pk)

where the 'accessible_obj_set' is either a reverse foreign key, or
some method that returns a queryset with apropriate filters to return
only the rows that a given user can access.

> If you can come up with a method that associates form data with a database row that doesn’t involve the PK, I would love to know. Keep in mind that most tables only have the one unique identifier.

there are many fans of using UUIDs as primary key. myself, i use them
for any user-visible record when there might be several instances of
the same code. this way i can merge or split datasets without
changing user-visible references.


--
Javier

[Carl Meyer]

Hi Peter,

Yes, this is a good clarification, thanks. What I meant to say is that
simply exposing the ID information to the user isn't necessarily a
security issue. But any server-side use of a client-provided ID (whether
provided in the URL or in a form field) needs to be properly validated
on the server side (just like any other data received from the client).
Formsets do this validation; if they didn't, they would be insecure.

My main point is that it's not at all adequate to say that "if a user
has access to edit one object, they probably have access to edit all the
others anyway." They may or they may not; that's application-specific.
Formsets allow the developer to specify exactly what queryset of objects
the user should be allowed to edit via the formset, and they validate
that the user can only edit those objects, and no others (even if the
hidden ID field is manually modified.)

> If you can come up with a method that associates form data with a
> database row that doesn’t involve the PK, I would love to know. Keep
> in mind that most tables only have the one unique identifier.

I'm not saying that you shouldn't use unique IDs to associate form data
with rows in the database (though there are some valid reasons to prefer
e.g. a UUID or a unique slug to an auto-incrementing ID in some cases).
I'm just saying that you should never trust an ID value (or any data)
coming from the client; you should ensure on the server side that the
user actually should have access to the object they are trying to access.

Javier showed a good method for doing this in typical cases, by passing
a limited queryset rather than a model class to get_object_or_404.

Carl

[Rob Ladd]

Carl Meyer said:

"Whatever queryset you pass to the model formset limits the 
available rows for editing. The end user can edit the PK to refer to any 
item in that queryset, but not any item in the table. "

That's not true, based on my observation. 

As long as the PK refers to any object of that type, it can be edited by monkeying with the hidden id field.

I think that's a pretty big gotcha, since we have conveniences like permission_required decorator, {% csrf_token %}, etc.

[Carl Meyer]

Hi Rob,

On 05/03/2016 12:13 PM, Rob Ladd wrote:
> Carl Meyer said:
>
> "Whatever queryset you pass to the model formset limits the
> available rows for editing. The end user can edit the PK to refer to any
> item in that queryset, but not any item in the table. "
>
> That's not true, based on my observation.
> As long as the PK refers to any object of that type, it can be edited by
> monkeying with the hidden id field.

I replied to this in the other thread you started:
https://groups.google.com/d/topic/django-users/tJpKWbij1B0/discussion

Short version, I can't reproduce the behavior you describe.

Carl