Problem between Pyramid's CSRF protection and Deform

[Laurent Daverio]

Hello List,

I'd like to report a problem I've just encountered, occurring betwen
Pyramid's CSRF protection and Deform.

Basically, I have a Pyramid 2.0 web app configured along the lines of
the "URL dispatch wiki tutorial"
(https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/tutorials/wiki2/authentication.html),
with some Deform forms in it.

The Deform Demo
(https://deformdemo.pylonsproject.org/pyramid_csrf_demo/) shows how to
use a deferred value to create hidden field "csrf_token" in the
generated forms.

But there's a problem: the token generated that way doesn't have the
same value as when I directly call get_csrf_token() in a template.

As I don't have the time/energy to fully investigate the problem right
now, I think I will just use a workaround: as I'm using Diazo as a
theming engine (awesome tech, btw), I think I will add a rule to
inject the token into every form. Should work.

Still, I wanted to take the time to report the problem, in case it
could be useful.

Laurent.

[Mikko Ohtamaa]

Hi Laurent,

The Deform Demo
(https://deformdemo.pylonsproject.org/pyramid_csrf_demo/) shows how to
use a deferred value to create hidden field "csrf_token" in the
generated forms

 

But there's a problem: the token generated that way doesn't have the
same value as when I directly call get_csrf_token() in a template.

Usually, this value is tied to the user session. Out of my head, without inspecting the code, I would suspect issues with, or mishandling of, cookies, sessions and such.

Br,

Mikko

[Steve Piercy]

It's difficult to say without your example. I've been using CSRF as shown in the Deform demo without any issues.

--steve

[Laurent Daverio]

Thank you Steve. I'll have to think about it, not that the code is
secret, just a matter of knowing what to post to be relevant.

Le mer. 28 avr. 2021 à 22:10, Steve Piercy
<steve.pi...@gmail.com> a écrit :

> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/44979a98-12ae-239e-8478-c2323aecfaf1%40gmail.com.

[Laurent Daverio]

Hi Mikko, thank you for your reply :)

I don't think I'm doing anything weird there. The problem happens in a
class-based view.
I can see the 'csrf_token' cookie qith the right value, I can display
the same value inside a template by calling get_csrf_token(), but the
value generated inside the deferred function is different, although
being passed (I think) the same request object...

> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAK8RCUuVJAJ6rAOrgmQ0W%2Bu_RVfo598oWU%3D5U_k-4JrvtOM7Cg%40mail.gmail.com.

[Laurent Daverio]

OK, I've been able to nail it down on a simple example : depending on
the CSRF storage policy I use, "request.session.get_csrf_token()"
(called from python or a template) and "get_csrf_token()" (called from
a template) return the same value *or not*.

- no storage policy => ok
- LegacySessionCSRFStoragePolicy => ok
- CookieCSRFStoragePolicy => ko

I'm attaching my example, I called it "onefile.py", although I needed
two files actually (one python file + one mako template). Sorry ;)

[Laurent Daverio]

So, if I follow this line of reasoning, the way to get the same value
as in the template is to use :

from pyramid.csrf import get_csrf_token
print get_csrf_token(request)

and *not* :

print request.session.get_csrf_token()

[Steve Piercy]

They are effectively the same.

https://docs.pylonsproject.org/projects/pyramid/en/latest/_modules/pyramid/csrf.html#LegacySessionCSRFStoragePolicy.get_csrf_token

In your code, you have configured two session factories. I assume you get the CSRF unique to each factory. ¯\_(ツ)_/¯

--steve

[Laurent Daverio]

No, I haven't configured two session factories, one of them is
commented out, so that I can test either.

If I use LegacySessionCSRFStoragePolicy, get_csrf_token(request) and
request.session.get_csrf_token() do return the same value. A look at
the source code shows they are implemented in the same way.

BUT it I use CookieCSRFStoragePolicy, they no longer return the same
value. That was my point.

As I was using the second policy in my code, as per the Pyramid
tutorial, I was always getting CSRF errors in my Deform forms. I've
solved the problem by using:

@colander.deferred
def deferred_csrf_default(node, kw):
request = kw.get('request')
return get_csrf_token(request)

instead of:

@colander.deferred
def deferred_csrf_default(node, kw):
request = kw.get('request')
return request.session.get_csrf_token()

Laurent.

[Janzert]

When using a pyramid.session session factory calling request.session.get_csrf_token seems to *always* be equivalent to using LegacySessionCSRFStoragePolicy (sort of by definition I suppose).

You can confirm this by looking at the session.get_csrf_token definition in

https://docs.pylonsproject.org/projects/pyramid/en/latest/_modules/pyramid/session.html#BaseCookieSessionFactory

and the call to it from LegacySessionCSRFStoragePolicy.get_csrf_token

https://docs.pylonsproject.org/projects/pyramid/en/latest/_modules/pyramid/csrf.html#LegacySessionCSRFStoragePolicy.get_csrf_token

Given the above, if you are using a different storage policy the request.session.get_csrf_token will (almost by definition) differ.

Janzert

[Jonathan Vanasco]

They're not the same at all.

The difference is on purpose.

Janzert is correct, though his description may not necessarily be clear.

The following might make more sense:

The two functions do the following:

  pyramid.csrf.get_csrf_token(request)

    discern active ICSRFStoragePolicy

    invoke {ICSRFStoragePolicy.get_csrf_token()}

    

  request.session.get_csrf_token()

    invoke {self.get_csrf_token()}

Because of that difference, the following happens.

  1. when using `LegacySessionCSRFStoragePolicy`, `SessionCSRFStoragePolicy`, or `None` (which is one of those IIRC):
      a) pyramid.csrf.get_csrf_token always uses the Session
      b) request.session.get_csrf_token is just a shortcut to the above
    

  2. when using `CookieCSRFStoragePolicy`:
      a) pyramid.csrf.get_csrf_token always uses the dedicated Session cookie, as you instructed it to.
      b) request.session.get_csrf_token is referencing a csrf_token within the session itself. Pyramid is not configured to ever look in the session for a csrf, because you told it to use a dedicated session cookie.

The `session.get_csrf_token` method is really a helper function for Pyramid and add-ons to have a standard interface for stashing a CSRF token in the session.  it might be worth changing those interface names to have leading underscores, so the public doesn't rely on them.  Or maybe add a warning docstring that `pyramid.csrf.get_csrf_token` should be used by developers.

In any event, you are getting different results because you are telling pyramid to use a cookie for the csrf token, then using one method that queries that cookie (correct value!) and a second method that queries the active session for a token -- which is not tied to the pyramid csrf system in any way.

[Laurent Daverio]

Hello Jonathan,

thank you for your message, and sorry for my late answer, I'm seeing
it only now. I've understood the difference by now, having spent a
number of hours on the problem (I'm not a Python or Pyramid newbie,
but I admit I am (or was?) a CSRF newbie).

> The `session.get_csrf_token` method is really a helper function for Pyramid and add-ons to have a standard interface for stashing a CSRF token in the session.
> it might be worth changing those interface names to have leading underscores, so the public doesn't rely on them. Or maybe add a warning docstring that
> `pyramid.csrf.get_csrf_token` should be used by developers.

Actually, when I set out upgrading my code to make it compatible with
Pyramid 2.x, I started with one of the official tutorials (SQLAlchemy
+ URL dispatch wiki tutorial), which says
(https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/tutorials/wiki2/definingviews.html#csrf-protection):

- Use CookieCSRFStoragePolicy
- Add a hidden field with "get_csrf_token()" in your templates

So, I was led to believe it was the right way. Erroneously, apparently...

Thanks for the clarification,

Laurent.

Le jeu. 13 mai 2021 à 00:34, 'Jonathan Vanasco' via pylons-discuss
<pylons-...@googlegroups.com> a écrit :

> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/58dd5c50-8c8a-442f-a2b1-2b6f1587d31an%40googlegroups.com.

[Steve Piercy]

Ah, yes, I see now. I was mistaken, too, which added to the confusion.

This tutorial and deformdemo use different methods and have different stacks. I don't know whether either is wrong, but perhaps just different? Perhaps bits of each implementation were mixed together during your exploration?

https://youtu.be/5sxRGSFEP30

This prompted me to do a little digging, and I found that although Deform has a schema.CSRFSchema class, it is documented only in the docstrings. I fixed that on both `main` and `2.0-branch`.

https://docs.pylonsproject.org/projects/deform/en/main/api.html#deform.CSRFSchema
https://docs.pylonsproject.org/projects/deform/en/latest/api.html#deform.CSRFSchema

There might be some inconsistencies between and within Deform, deformdemo, and Pyramid in the examples. Deform has not had as consistent maintenance as Pyramid. If you spot any issues, please let us know, and we'll try to resolve them. I'd be open to adding more examples to deformdemo.

--steve