Running pending migrations in Middleware

[David Grant]

Anyone see any problems with running migrations in Middleware?

import logging
import os
import time

from django.core.management import call_command
from django.http import HttpResponse

from products.models.models import AppVersion

LOG = logging.getLogger(__name__)


# Used for testing locally when the GAE_VERSION environment is not set
FAKE_VERSION_TESTING = 8


class MigrateMiddleware:

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        start_time = time.time()
        try:
            LOG.info("Running migrate middleware")
            # Figure out what version of our app is now deployed            
            app_version, created = AppVersion.objects.get_or_create(version=os.getenv('GAE_VERSION', FAKE_VERSION_TESTING))
            if not app_version.migrations_succeeded or created:
                # Previous attempt at this deployed version failed, or this was a new deployment.
                try:
                    call_command('migrate')
                except:
                    return HttpResponse("Running a Django migration failed! Investigate.", status=500)
                app_version.migrations_succeeded = True
                app_version.save()
        finally:
            LOG.info("Took: %ss", time.time() - start_time)
        return self.get_response(request)

It seems to work just fine in a quick test.

[Markus Holtermann]

Yes, I do see several problems. Some of them:

- All middleware are run for every request
- What you're doing can slow down your request response time to minutes, depending on home many migrations you have
- Depending on your database, migrations might not be atomic thus causing conflicts
- You have no way to intervene when something goes south
- ...

I can elaborate in more detail when you want. But please, don't run migrations as part of the request-response cycle.

/Markus

> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/bdfd6393-6d68-4285-b5a6-eb1be98ef467%40googlegroups.com <https://groups.google.com/d/msgid/django-users/bdfd6393-6d68-4285-b5a6-eb1be98ef467%40googlegroups.com?utm_medium=email&utm_source=footer>.

[James Schneider]

On Sun, Jul 28, 2019, 12:47 AM David Grant <david...@gmail.com> wrote:

Anyone see any problems with running migrations in Middleware?

What's the actual problem you are trying to solve by doing this? This is a bad idea for a number of reasons, and I can't think of any good ones.

-James

[David Grant]

The reason I want to do this by the way, is because I'm using Google AppEngine and I have no access to the server. The alternative is to have the exact same code checked out on my local machine. Connect to the remote database with a special proxy tool that google provides (so the db becomes available on a local port) and then run the migrations. Problems with this are: 1) Hitting the remote database via the proxy is extremely slow, like ridiculously slow and 2) there is a chance I have the wrong local code. ie. this could mean I run some extra migrations or not run enough migrations, thus causing production to be broken.

Thanks for bringing up those points.

All middleware are run for every request 

Yep, this is an issue if "migrate" is called on every request. I solved this by recording whether or not migrations have been run or not on the current version of the app, and also whether it was successful or not. So the only hit I take on every request one query, which will be cached in the db and could also be cached in memory if I really wanted to. Currently, with just the one query it's like 10ms when I do it locally. Not the bottleneck.

What you're doing can slow down your request response time to minutes, depending on how many migrations you have.

What's the alternative? Put the app in a maintenance mode? Either way the app will be unusable for a bit. I guess there is an issue where the requests could exceed the maximum request time (in my case in AppEngine it is 60 seconds) and so if the migration takes longer that, that would be a problem. I could kick off migration asynchonously?

Depending on your database, migrations might not be atomic thus causing conflicts 

I'm using Postgresql so my understanding is my migrations can all run atomically.

> an email to django...@googlegroups.com.

[David Grant]

I'm using Google AppEngine and I have no access to the server. The alternative is to have the exact same code checked out on my local machine. Connect to the remote database with a special proxy tool that google provides (so the db becomes available on a local port) and then run the migrations. Problems with this are: 1) Hitting the remote database via the proxy is extremely slow, like ridiculously slow and 2) there is a chance I have the wrong local code. ie. this could mean I run some extra migrations or not run enough migrations, thus causing production to be broken.

Ideally I would just run `./manage.py migrate` on the server, but that's just not possibe.

Also a colleague of mine has been doing this for a Java app for years. There's a quick check in middleware for current db version and then it runs migrations if necessary.

[David Grant]

On Sunday, July 28, 2019 at 2:35:03 PM UTC-7, David Grant wrote:

I'm using Google AppEngine and I have no access to the server. The alternative is to have the exact same code checked out on my local machine. Connect to the remote database with a special proxy tool that google provides (so the db becomes available on a local port) and then run the migrations. Problems with this are: 1) Hitting the remote database via the proxy is extremely slow, like ridiculously slow and 2) there is a chance I have the wrong local code. ie. this could mean I run some extra migrations or not run enough migrations, thus causing production to be broken.

I guess I can do this as part of the deploy, which makes the version is guarenteed to be the correct one. But I still need to use the google cloudsql proxy which sucks.