#3527 - better debug traceback with code executing...

[jedie]

Why has django not a interactive AJAX traceback debugger?

Using a interactive debugger you can play with objects at any point in
the error traceback.

I known, a web shell is a open security hole. But the interactive
debugger should only running with the development Web server and if
debugging is on.
The development server is not for production use. So there is IMHO no
problem.

Here a example pictures from Pylons:
http://pylonshq.com/img/screenshots/doctraceback.gif

There exist some very nice modules:
http://pylonshq.com/docs/interactive_debugger.html
http://pythonpaste.org/module-paste.evalexception.html
http://trac.pocoo.org/browser/colubrid/trunk/colubrid/debug.py

See also:
http://blog.ianbicking.org/ajaxy-exception-catching.html

Here you can see a screencast, how paste evalexception works:
http://pythonpaste.org/screencasts/evalerror-screencast.html

Existing django ticket: http://code.djangoproject.com/ticket/3527

[Malcolm Tredinnick]

Hi,

On Mon, 2007-04-09 at 04:33 -0700, jedie wrote:
> Why has django not a interactive AJAX traceback debugger?
>
> Using a interactive debugger you can play with objects at any point in
> the error traceback.
>
> I known, a web shell is a open security hole. But the interactive
> debugger should only running with the development Web server and if
> debugging is on.
> The development server is not for production use. So there is IMHO no
> problem.

[...]

> Existing django ticket: http://code.djangoproject.com/ticket/3527

The reason I originally closed that ticket as "wontfix" -- and I still
think it's the right reasoning -- is because the debug traceback handler
is not associated with whether or not the development server is running.

Instead, it is triggered by whether or not DEBUG is True. Sometimes you
want to have DEBUG=True in production environments, whether for just a
little period of time -- to debug something -- or for longer. So I am
reluctant to put in something that might be a security hole if there's
any chance of it being run on a production site.

Regards,
Malcolm

[Armin Ronacher]

Hi,

On Apr 9, 1:48 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

> Instead, it is triggered by whether or not DEBUG is True. Sometimes you
> want to have DEBUG=True in production environments, whether for just a
> little period of time -- to debug something -- or for longer. So I am
> reluctant to put in something that might be a security hole if there's
> any chance of it being run on a production site.

The debugger must be additionally of course. Something like
ENABLE_DEBUGGER = True/False in the settings. I would say the debugger
is something that is missing in django. But it should be optional.

Regards,
Armin

[Ned Batchelder]

If the only concern here is that debugging is a powerful feature that needs to be carefully controlled, then surely a setting to enable it is the right way to go?  After all, many security experts will tell you that the traceback handler we have now is a security hole, not because it lets you execute code, but because it reveals inner workings of the server, which can expose vulnerabilities.

Production servers need to have their settings carefully set.  That is true now, and it will be true if we add a DEBUGGER=True setting to enable this more powerful feature.  I say we add the power and let the administrators control where it appears.

--Ned.

-- 
Ned Batchelder, http://nedbatchelder.com

[Armin Ronacher]

Hoi,

Alright. Patch here: http://pocoo.org/~mitsuhiko/django_debugger.patch
Screenshot here: http://pocoo.org/~mitsuhiko/djangodebug.png

(Note, I edited those files in the same checkout I did the patches for
the __loader__ hook. So the patch is a bit bigger. Sorry for that)

Enable by adding ENABLE_DEBUGGER = True into the settings.py file

Also note: I had to modify the CommonMiddleware. That's not the best
way to hook the traceback AJAX call in, maybe there are better ways.

Regards,
Armin

[Michael Radziej]

Malcolm Tredinnick schrieb:

> Hi,
>
> On Mon, 2007-04-09 at 04:33 -0700, jedie wrote:
>> Why has django not a interactive AJAX traceback debugger?
>>
>> Using a interactive debugger you can play with objects at any point in
>> the error traceback.
>>
>> I known, a web shell is a open security hole. But the interactive
>> debugger should only running with the development Web server and if
>> debugging is on.
>> The development server is not for production use. So there is IMHO no
>> problem.

I'm sure we could find a solution to make sure that it's not in use on a
production server. For example (in order of inceasing annoyment):

- use a separate setting to enable it (as has been suggested)
- use INTERNAL_IPS to limit who is able to use it
- require authorization
- make it available only with 'manage.py runserver', and not via
modpython or fcgi

I would find this sooooo useful.

Malcolm, would some of this additional protection help?

Michael

[Marek Kubica]

Hello,

On Mon, 09 Apr 2007 12:19:18 -0700, Armin Ronacher wrote:

> Alright. Patch here: http://pocoo.org/~mitsuhiko/django_debugger.patch
> Screenshot here: http://pocoo.org/~mitsuhiko/djangodebug.png

That's definitely cool. There are cases in which you'd like to have a
prompt to check what went wrong. Feels a little bit lispy, I like it.

> Enable by adding ENABLE_DEBUGGER = True into the settings.py file

I'd really prefer to use some other way than an additional setting. How
about something to detect whether it runs in the development server?

> Also note: I had to modify the CommonMiddleware. That's not the best way
> to hook the traceback AJAX call in, maybe there are better ways.

I hope so. Not all people use CommonMiddleware. Or maybe it really should,
but then it should be documented that the debugger is only active with
CommonMiddleware.

But generally speaking - having a prompt built in is definitely cool and
useful for debugging.

regards,
Marek

[Marek Kubica]

Hi,

I'm bringing this question up again, as no decision has been made.

The current state is:
- A AJAXy debugger could be useful. Some other frameworks have a
debugger, so it's not a useless gimmick.
- Having a debugger in the traceback is dangerous. We all agree
- Having the debugger enabled only in some cases (running only in
Django's own development server or enabled via a setting or only for
certain IPs, or only if the user is authentificated) could be a
feasible way to go
- We have a patch by Armin Ronacher, which I uploaded to the trac

I'd be happy to have a conclusion. Is this really a wontfix decision or
can the debugger be included, provided it has enough security precautions?

regards,
Marek

[Baptiste]

I am +10 on this patch.
If this debugger was disabled if DEBUG is False, that wouldn't solve
the problem ? I mean, are they people who use DEBUG mode in
production ?

[Baptiste]

Oh, and maybe we can check if request.user.is_authenticated() and
request.user.is_staff() before enabling it ?

[SmileyChris]

Or even require a superuser?

[Karen Tracey]

But don't you want to be able to debug the code paths that run for all requests, not just authenticated/staff/superuser? 

[Baptiste]

Hum, yeah, so maybe we could add a permission that could be set to a
lambda user and that would allow the access to this debugger. It
wouldn't care if the account is disabled or not, so that could
simulate an unauthenticated user (if the app checks
user.is_authenticated() and user.is_active). I know, not simple, but
that is true, we may want to simulate the behaviour of the app for
everyone... maybe the syncdb of the admin that creates a superuser
could create a debugger user ?

On 15 avr, 23:26, "Karen Tracey" <kmtra...@gmail.com> wrote:
> But don't you want to be able to debug the code paths that run for all
> requests, not just authenticated/staff/superuser?
>

> On 4/15/07, SmileyChris <smileych...@gmail.com> wrote:
>
>
>
> > Or even require a superuser?
>
> > On Apr 16, 6:00 am, "Baptiste" <baptiste.gou...@gmail.com> wrote:
> > > Oh, and maybe we can check if request.user.is_authenticated() and

[Karen Tracey]

Sounds complicated, and I don't think it's necessary.  Malcolm's concern was that this special debugger never be enabled in production.  What user is associated with a request really has nothing to do with whether the code is running in production -- in order to test out changes in my app before they go live I need to test running without a login, with a non-admin login, and with an admin login, all of which I do under the development server.

The key for whether this debugger can be activated  seems to be that the code is running under the development server.  I don't know if there is any way to easily check that?  I suppose you might want a specific setting as well to enable/disable it even for that limited environment, if there's a chance people would not want it active, or if there's a chance people are using the development server in production (does anyone do that?)....

Karen

[Armin Ronacher]

Hoi,

-1 for the database interaction. The debugger should also work if the
database is fucked up.

Regards,
Armin

[Baptiste]

Maybe some people use the development server in production... and
maybe some people don't use it in devel !
Precarious to take that for reference, don't you think ?

On 16 avr, 01:45, "Karen Tracey" <kmtra...@gmail.com> wrote:
> Sounds complicated, and I don't think it's necessary. Malcolm's concern was
> that this special debugger never be enabled in production. What user is
> associated with a request really has nothing to do with whether the code is
> running in production -- in order to test out changes in my app before they
> go live I need to test running without a login, with a non-admin login, and
> with an admin login, all of which I do under the development server.
>
> The key for whether this debugger can be activated seems to be that the
> code is running under the development server. I don't know if there is any
> way to easily check that? I suppose you might want a specific setting as
> well to enable/disable it even for that limited environment, if there's a
> chance people would not want it active, or if there's a chance people are
> using the development server in production (does anyone do that?)....
>
> Karen
>

[Russell Keith-Magee]

On 4/16/07, Armin Ronacher <armin.r...@active-4.com> wrote:

Hoi,

-1 for the database interaction. The debugger should also work if the
database is fucked up.

Mind the language, please. This is a formal mailing list discussing serious issues. Keeping the language civil is just one way we ensure that the discussion maintains the seriousness it deserves.

Yours,
Russ Magee %-)

[Karen Tracey]

On 4/16/07, Baptiste <baptist...@gmail.com> wrote:

Maybe some people use the development server in production... and
maybe some people don't use it in devel !
Precarious to take that for reference, don't you think ?

True.  Tying the special debugger to the development server leaves people who develop under apache or whatever out in the cold.  So I think that leads to a setting, somthing other than DEBUG (since Malcolm's concern is that DEBUG might need to be on in production settings from time to time), with a big fat disclaimer like the one for django.views.static.server.

My real concern is that it not be tied to the user associated with the request. You need to be able to debug code paths for all users during development.

Karen

[Simon G.]

My preference would be to have it as either a package in contrib, or a
third party package that users can install. I don't know how easy it
would be to override Django's default debug handler but shouldn't be
too hard. This way we don't have any on-by-default situations, and it
should be fairly safe since it'll have to be installed by a developer
(plugged into settings.INSTALLED_APPS) , and most people don't develop
with the same settings file that they run a production site off.

--Simon

[Baptiste]

And that other setting, even to True, won't be used if debug is to
False (to prevent oblivions) ?

Moreover, to stop lambda users from accessing this debugger, maybe we
could enforce the admin to type a password in the console before
proposing its completes features ?
I mean, you have the debug text input, but whatever you type, if you
have not entered something like 'e1desFrde2D', the output is 'Please
enter the debug password'.

On 16 avr, 14:10, "Karen Tracey" <kmtra...@gmail.com> wrote:

[Armin Ronacher]

Hoi,

A possible solution would be moving the debug view from the core into
a middleware. Then it's possible to replace the middleware and move
the debugger into a contrib package.

Regards,
Armin

[jedie]

Whats about INTERNAL_IPS [1] ?

The special debugger is only available if...
...settings.DEBUG is ON (or a seperate Variable)
and
...the request IP is in INTERNAL_IPS

[1] http://www.djangoproject.com/documentation/settings/#internal-ips

[Jacob Kaplan-Moss]

On 5/15/07, jedie <google...@jensdiemer.de> wrote:
> Whats about INTERNAL_IPS [1] ?

I think you need to let this one go. Malcom's already given a -1 and
no other developers stepped up with even a +0.

I'd love to see you develop this as an add-on -- a piece of exception
middleware, probably -- but I don't think it's appropriate for Django
core.

Jacob

[Michael Radziej]

On Thu, May 17, Jacob Kaplan-Moss wrote:
> On 5/15/07, jedie <google...@jensdiemer.de> wrote:
> > Whats about INTERNAL_IPS [1] ?
>
> I think you need to let this one go. Malcom's already given a -1 and
> no other developers stepped up with even a +0.

Malcolm only gave it -1 for a very specific reason, and that was:

On Mon, Apr 09, Malcolm Tredinnick wrote:
> The reason I originally closed that ticket as "wontfix" -- and I still
> think it's the right reasoning -- is because the debug traceback handler
> is not associated with whether or not the development server is running.

So I consider it valid to bring up other ideas.

After leaving this hang around in my head for a while, I'm

a) +0 for a combination of INTERNAL_IPS and a specific setting
b) +0.5 for a specific setting that lists the IPs that are allowed
connect to the debugger.
c) +1 for the same as b) implemented in a contrib middleware.

My reason is:

During debugging, I often need to see attributes of data which are not
displayed, or the result of fuctions. Besides, it is a real good tool for
beginners. But I really don't want anybody else to be able to fiddle with
the debugger view. Thus, I really want to set which IP is allowed. Debugging
usually happens within a safe and controlled network, so I can trust the IP.

The traceback handler isn't something that has seen a lot of changes, so I
guess the maintenance for this would be minimal. A middleware would still be
a better separation.

Michael

--
noris network AG - Deutschherrnstraße 15-19 - D-90429 Nürnberg -
Tel +49-911-9352-0 - Fax +49-911-9352-100
http://www.noris.de - The IT-Outsourcing Company

Vorstand: Ingo Kraupa (Vorsitzender), Joachim Astel, Hansjochen Klenk -
Vorsitzender des Aufsichtsrats: Stefan Schnabel - AG Nürnberg HRB 17689