Pyramid maintenance mode

[C J]

Hi everybody,

I am looking for an easy way to temporarily redirect all the users of my pyramid website to a maintenance vue without having to comment/delete, etc my routes.
I would like to make it easy to re-activate the others routes when the maintenance is done.
I found this :
https://pypi.org/project/pyramid_maintenance/
but  I always get :

in renderer

    raise ValueError('No such renderer factory %s' % str(self.type))

ValueError: No such renderer factory .jinja2"

with my browser displaying :
"Internal Server Error The server encountered an unexpected internal server error (generated by waitress)"

I am new to Pyramid so please give me the necessary details step by step.

Best regards.
Cedric J.

[Jonathan Vanasco]

I typically handle this on nginx which sites in front of Pyramid.  if you wanted to do everything in python, you could probably use WSGI middleware to route to a separate maintenance application or html file.

[Thierry Florac]

I've built a custom Pyramid tween to handle this and redirect requests while application is up!

It can handle redirects (based on regular expressions) before or after the request is handled by Pyramid application, to be able to set the site in "maintenance mode", or to handle custom redirects in case of NotFound exceptions...

All configuration is done through our Pyramid CMS.

Best regards,

Thierry

-- 

  https://www.ulthar.net -- http://pyams.readthedocs.io

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/98a779a5-5bcc-4971-a271-a202cc49f732n%40googlegroups.com.

[Jonathan Vanasco]

i should add, the pattern for doing this in nginx is often called a flag or semaphore

i found this article that goes through a how-to  https://blog.devcloud.hosting/configuring-nginx-for-quickly-switching-to-maintenance-mode-e4136cf497f3

basically, you just touch/remove a specific file to change the routing on nginx. while the existence of the file is checked on every request, due to the way nginx and operating system cache the information this is negligible and essentially handled in memory.

[C J]

That's really interesting Thierry. Can you please show me how to do? 
I have tried to use tweens. I tried many things starting with
pyramid.tweens = pyramid_maintenance.tween_maintenance and modifying the __init__.py file, however, I do not understand what they are and how to use them despite hours spent reading the Pyramid documentation and different articles.

Being working on a website that is already in production I also wonder how I would be able to implement your solution based on an external CMS.

Sender notified by Mailtrack 07/01/21 à 21:59:31

You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/jKTnofibd00/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBO5j0im6r0RRKEf%3D%3DzXyx5y_Qp%3DUJ5bntEGQtdiDejKQ%40mail.gmail.com.

[Mike Orr]

I have a request-logging tween in only a page of code. It's straightforward to write from the documentation.

https://pyramid.readthedocs.io/en/latest/narr/hooks.html#registering-tweens

I adapted the code for your use case (untested). Your module would have something like this:

# Module 'myapp.lib.mytween'

def includeme(config):

    # Calculate the dotted name of the factory function.

    # E.g., "myapp.lib.mytween.my_tween_factory".

    factory_name = __name__ + ".my_tween_factory"

    config.add_tween(factory_name)

    # You can wrap app this in an 'if' to conditionally enable it; e.g.,

    # ``if pyramid.settings.asbool(config.registry.settings.get("myoption", True))``

def my_tween_factory(handler, registry):

    # Return a tween callable.

    # 'handler' is the next tween or the WSGI application.

    # Deployment settings are in 'registry.settings'.

    def my_tween(handler, request):

        if is_maintenance_mode:   # Could limit it to certain request.path's.

            # Return an error response, bypassing the application.

            return pyramid.response.HTTPServiceUnavailable(maintenance_message)

        else:

            # Call next tween/application and return its response unchanged.

            return handler(request)

Then list your module in 'pyramid.includes' in the config file. E.g., "myapp.lib.mytween".

That 'if is_maintenance_mode' condition could check whether a specified file exists. The file path could be in a config setting.

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAJgq6FJC11OfFNbfhhPwfMKRJsJfpd2UpWUDs5SAm5hKaMWpzA%40mail.gmail.com.

--

Mike Orr <slugg...@gmail.com>

[Mike Orr]

I forgot the last line. The end of 'my_tween_factory' needs to 'return my_tween'.

--

Mike Orr <slugg...@gmail.com>

[C J]

To everybody: thanks a lot for your answers. Sorry for the delay: I am living in France.
Bay the way I am still interested in the Pyramid CMS solution steps. 

Mike,
I have created a file named maintenance.py in my utils module.
It contains:

import pyramid

def includeme(config):

  # Calculate the dotted name of the factory function.

  # E.g., "myapp.lib.mytween.my_tween_factory".

  maintenance_tween = __name__ + ".maintenance_tween_factory" 

  config.add_tween(maintenance_tween)

  # You can wrap app this in an 'if' to conditionally enable it; e.g.,

  # ``if pyramid.settings.asbool(config.registry.settings.get("myoption", True))``

def maintenance_tween_factory(handler, registry):

  # Return a tween callable.

  # 'handler' is the next tween or the WSGI application.

  # Deployment settings are in 'registry.settings'.

  def my_tween(handler, request):

    is_maintenance_mode = request.registry.settings["in_maintenance"]

     if is_maintenance_mode and not "dbmigration" in request.matchdict: # Could limit it to certain request.path's.

      # Return an error response, bypassing the application.

       return pyramid.response.HTTPServiceUnavailable(

         {

           "Maintenance": "Please note that we will be performing important server maintenance in a few minutes, during which time the server will be unavailable. If you are in the middle of something important, please save your work or hold off on any                  critical actions until we are finished."

         }  

       )  

     else:

       # Call next tween/application and return its response unchanged.

       return handler(request)

  return maintenance_tween

I have also added    in_maintenance = True       in my production.ini file.

• Now where should I import this modules? Where should I call these functions?
• Another point is that maintenance_tween  is unknow in th context of  def maintenance_tween_factory(handler, registry) so will return maintenance_tween  work?
• Also I want a Cornice API (dbmigration) which is in my views module to still be available in production when I send a POST request or a GET request to /dbmigration during the maintenance time. 

I have read https://pyramid.readthedocs.io/en/latest/narr/hooks.html#registering-tweens
But I do not understand very well where to write and call the tweens lines of code.

[C J]

I fixed my code by returning the function inside. But I still need to understand where to call theses functions to set everything up.  

def maintenance_tween_factory(handler, registry):

# Return a tween callable.

# 'handler' is the next tween or the WSGI application.

# Deployment settings are in 'registry.settings'.

def maintenance_tween(handler, request):

is_maintenance_mode = request.registry.settings["in_maintenance"]

if is_maintenance_mode and not "dbmigration" in request.matchdict: # Could limit it to certain request.path's.

# Return an error response, bypassing the application.

return pyramid.response.HTTPServiceUnavailable(

{

"Maintenance": "Please note that we will be performing important server maintenance in a few minutes, during which time the server will be unavailable. If you are in the middle of something important, please save your work or hold off on any critical actions until we are finished."

}

)

else:

# Call next tween/application and return its response unchanged.

return handler(request)

return maintenance_tween

[Thierry Florac]

Hi Cedric,

You just have to include your package from Pyramid main configuration file, using the "includes" section...

The "includeme" function will then be called automatically to register your tween!

Thierry

-- 

  https://www.ulthar.net -- http://pyams.readthedocs.io

To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/e72f1fc0-a202-4244-8e57-576053b68ce2n%40googlegroups.com.

[C J]

Thank you!
I think I finally got it. Now I would like to allow only 2 routes:

• one is my "maintenance" because I want to use a jinja template instead of returning the HTTPException
• one is "dbmigration" because I still need to call my Cornice API

For the moment the maintenance mode affects all my routes and when its value is evaluated and there is a second problem which is an infinite loop when is_maintenance_mode is not True, because of

else:

  # Call next tween/application and return its response unchanged.

  return handler(request)

How should I do to avoid these issues?

 
 

Le vendredi 8 janvier 2021 à 14:34:53 UTC+1, C J a écrit :

I get 'No module named 'maintenance_tween'. Should I separate the two functions in 2 different files, both of them being in the same module?

[C J]

I try to understand why my "or not" have no effect:

def maintenance_tween_factory(handler, registry):

    # Return a tween callable.

    # 'handler' is the next tween or the WSGI application.

    # Deployment settings are in 'registry.settings'.

    def maintenance_tween(request):

        #print(request)

        is_maintenance_mode = registry.settings.get("in_maintenance")

        if is_maintenance_mode == "True":

            print(request.url)

            print("maintenance" in request.url) # True but still affected by the maintenance mode

            url = request.route_url('maintenance')

            if \

            not "/api/dbmigration" in request.url \

            or not "maintenance" in request.url:

                return HTTPServiceUnavailable(

                    "Please note that we will be performing important server maintenance in a few minutes, during which time the server will be unavailable. If you are in the middle of something important, please save your work or hold off on any critical actions until we are finished."

                     )

                #return HTTPServiceUnavailable(location=url)

        # # Could limit it to certain request.path's.

        # if not ("/api/dbmigration" in request.url and is_maintenance_mode == "True") \

        #         or not ("maintenance" in request.url and is_maintenance_mode == "True"):

        #     # Return an error response, bypassing the application.

        #     return HTTPServiceUnavailable(

        #             "Please note that we will be performing important server maintenance in a few minutes, during which time the server will be unavailable. If you are in the middle of something important, please save your work or hold off on any critical actions until we are finished."

        #         )

[Mike Orr]

The includeme function takes care of all the tween registration for
you, so you just have to include the module using the
"pyramid.includes" setting or such. You've got that working.

Since you're enabling maintenance mode with a config setting, and
you'll have to restart the application to change it, you can hoist the
'in_maintenance' setting check into the includeme function. E.g.,

if registry.settings.get("in_maintenance", False):
tween = __name__ + ".maintenance_tween_factory"
config.add_tween(tween)

That way the tween will be registered only if maintenance mode is
enabled. That will simplify your tween code slightly and is one
millisecond more efficient.

As for your other problems of the maintenance screen never appearing,
or not appearing with the right URLs (I'm not sure which), I would add
some print or logging statements in the tween function to confirm it's
being called and see what its request.url, request,matchdict, and
request.path are. (I prefer request.path over request.url.) They may
be different from what you expect, and if so we'd need to figure out
why.

I would also simplify the 'if not ... or not ...' expressions. While I
don't see anything in particular wrong with them, there may be a logic
error buried in the expressions somewhere, and separating them out and
printing the intermediate values may reveal what it is

> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/3ea70938-5f98-4a6b-97ba-25dea5edc50dn%40googlegroups.com.



--
Mike Orr <slugg...@gmail.com>

[Mike Orr]

Don't forget the 'asbool()'. Without it the 'in_maintenance' text
setting value may be interpreted wrong. E.g., '0' or ' ' or 'false'
would evaluate to True.

--
Mike Orr <slugg...@gmail.com>

[C J]

Thank you Mike!

I had forgotten "asbool()".
It takes time locally to load my page but I finally allow specific route while having the maintenance mode activated.

handler(request) takes time. 

I consider that my questions are answered and that  my problem is  solved. 

Thank you very much everybody: you helped me a lot. It is great to have found this Google Group.
Have a nice day!