System check framework

[Christopher Medrela]

Hello!

This summer I'm a student of Google Summer of Code. I was working at

introducing a new system check framework (merging django-secure was a part of

my proposal, but I didn't manage to do it due to my time overestimation). The

project is coming to the end and there is a pull request [1] that is almost

ready for merging. I would like to present you with the capabilities of the

framework as well as a high-level overview. I would like to hear your opinion

how the framework can be improved -- this is a last chance for that! This

thread is a continuation of old "[GSoC] Revamping validation framework and

merging django-secure once again" thread [2].

[1] https://github.com/django/django/pull/1364

[2] https://groups.google.com/forum/#!topic/django-developers/fEf21dtpqDE

Here I would like to say thank you to my mentor, Russell Keith-Magee, who gave

me a lot of advices, kept an eye at my project, did reviews and supported me.

Preston Holmes also spent a lot of time reviewing code. Many thanks to

numerous other contributors!

Now let me introduce you the framework.

Overview

I've introduced a new system check framework that replaces the old validation

framework performing mainly model validation. The new framework is responsible

for model validation, app-specific checks like ModelAdmin validation as well

as compatibility checks. It introduces concept of warnings. Warnings as well

as errors can be silenced. The framework is open-ended which means that you

can register your own check stuff.

Messages

The framework uses a concept of messages similar to messages from message

framework or logging framework. A message can be a light message like a

warning, info or debug; or a serious message, e.g. an error or critical

(error). Every message consists of four attributes:

• required error message (`msg`);
• optional `hint`; in order to force developers to think carefully if they
  cannot provide any useful hint, they need to explicitly pass hint=None in
  that case;
• optional invalid object (`obj`);
• optional unique message identifier (`id`) used to refer to messages, i.e.
  "E001" (a core error) or "admin.W001" (a warning issued by admin app).

Performing checks

In order to perform all kinds of checks, type:

    python manage.py check

This command does the same stuff as the old `validate` and `check` commands.

`validate` is now deprecated and delegates to `check` command.

You can still validate some specific apps:

    python manage.py check admin auth

Or you can run a subset of checks labeled with a given tag:

    python manage.py -t compatibility  # alternative: --tag compatibility

Silencing errors/warnings

A new setting called SILENCED_SYSTEM_CHECKS was introduced. It's a list of

message identifiers. If you put on the list a light message like a warning, it

won't be printed anymore. Putting on the list a serious message like an error

won't hide the message, but will allow you to i.e. run server.

Registering your own stuff

If you want to add a new check, you need to write a function that accepts

`apps` and `**kwargs` arguments where `apps` is a list of applications that

should be validated (or None if all apps should be validated). The function

should return a list of messages, even if it's an empty list. Finally, you

need to register this entry point. Let's look at this snippet implementing

some security checks:

    from django.conf import settings

    from django.core import checks

    # Label this entry point with 'security' tag. This tag can be used while

    # invoking check command, e.g. you can write `./manage.py -t security`.

    @checks.register('security')

    def my_security_check(apps, **kwargs):

        # This check does not perform any app-specific checks, so if we are

        # asked to validate some specific apps, we can skip checks.

        if apps is not None:

            return []

        if len(settings.SECRET_KEY) < 64:

            return [

                checks.Warning(

                    'Your SECRET_KEY is too short.',

                    hint=None,  # need to explicitly pass hint even if you cannot provide any hint

                    obj='settings.SECRET_KEY',

                    id='security.W001',

                )

            ]

        else:

            return []

Field/model/manager checks

You don't have to register your check stuff if it's called from another piece

of code that is registered (directly or indirectly). This is true especially

in the case of fields, models and managers. Each of these object have `check`

method (or classmethod in the case of models) that can be overriden in order

to add some validation stuff. Consider that you implemented a field that

accepts only values from [min, max] range and you want to check if min < max.

Here is a snippet of code:

    from django.core import checks

    from django.db import models

    class RangedIntegerField(models.IntegerField):

        def __init__(self, min=None, max=None, **kwargs):

            super(RangedIntegerField, self).__init__(**kwargs)

            self.min = min

            self.max = max

        def check(self, **kwargs):

            errors = super(RangedIntegerField, self).check(**kwargs) # (1) call the superclass

            # (2) Do some custom checks and add messages to `errors`:

            errors.extend(self._check_min_max_values(**kwargs))

            return errors # (3) return all messages

        def _check_min_max_values(self, **kwargs):

            if (self.min is not None and

                    self.max is not None and

                    self.min > self.max):

                return [

                    checks.Error(

                        'min greated than max.',

                        hint='Lower min or upper max.',

                        obj=self,

                        id='myapp.E001',

                    )

                ]

            return [] # When no error, return an empty list

[Carl Meyer]

Hi Christopher,

On 09/20/2013 02:07 PM, Christopher Medrela wrote:
> This summer I'm a student of Google Summer of Code. I was working at
> introducing a new system check framework (merging django-secure was a
> part of
> my proposal, but I didn't manage to do it due to my time
> overestimation). The
> project is coming to the end and there is a pull request [1] that is almost
> ready for merging. I would like to present you with the capabilities of the
> framework as well as a high-level overview. I would like to hear your
> opinion
> how the framework can be improved -- this is a last chance for that! This
> thread is a continuation of old "[GSoC] Revamping validation framework and
> merging django-secure once again" thread [2].
>
> [1] https://github.com/django/django/pull/1364
> [2] https://groups.google.com/forum/#!topic/django-developers/fEf21dtpqDE
>
> Here I would like to say thank you to my mentor, Russell Keith-Magee,
> who gave
> me a lot of advices, kept an eye at my project, did reviews and
> supported me.
> Preston Holmes also spent a lot of time reviewing code. Many thanks to
> numerous other contributors!
>
> Now let me introduce you the framework.

I haven't had the time this summer to keep up with your progress, but
based on your summary here, this looks really excellent! Thanks for your
work.

Carl

[Russell Keith-Magee]

Hi Chris,

On Sat, Sep 21, 2013 at 4:07 AM, Christopher Medrela <chris....@gmail.com> wrote:

Hello!

This summer I'm a student of Google Summer of Code. I was working at

introducing a new system check framework (merging django-secure was a part of

my proposal, but I didn't manage to do it due to my time overestimation). The

project is coming to the end and there is a pull request [1] that is almost

ready for merging. I would like to present you with the capabilities of the

framework as well as a high-level overview. I would like to hear your opinion

how the framework can be improved -- this is a last chance for that! This

thread is a continuation of old "[GSoC] Revamping validation framework and

merging django-secure once again" thread [2].

[1] https://github.com/django/django/pull/1364

[2] https://groups.google.com/forum/#!topic/django-developers/fEf21dtpqDE

Here I would like to say thank you to my mentor, Russell Keith-Magee, who gave

me a lot of advices, kept an eye at my project, did reviews and supported me.

Preston Holmes also spent a lot of time reviewing code. Many thanks to

numerous other contributors!

Let me repeat publicly what I said to you on our last call -- Thank *you* for your efforts over the summer. It was a pleasure to work with you, and you deserve to be proud of what you've produced. 

It's certainly a pity that we didn't get to merging django-secure, but the codebase you've provided gives us a very strong foundation on which to do that merge as a post-GSoC activity.

So - now the pressure is on me to do a final review and commit! In the meantime, I'd encourage anyone with an interest to look over the pull request and provide feedback.

Yours,

Russ Magee %-)