Gitpython push does work with modwsgi ... same code runs under Flask runserver

[Joel Weinfeld]

Hi -

I am having a problem with apache using mod_wsgi and python 3.6 on RedHat 7.2 using the gitpython package .... specifically the push ...

I get no errors ... the server that I am pushing to ...  receives no push ... the result from the push is:

     2018-06-08 13:03:17,584 - ProcessGit - INFO - push:[]

Documentaion states:

If the operation fails completely, the length of the returned IterableList will
be null."""

The same code on the same machine running through Flask runserver ... Works

2018-06-08 11:48:01,593 - ProcessGit - INFO - push:[<git.remote.PushInfo object at 0x7f1c3cf34518>]

The call that I am using is:

           .....

      repo_origin = repo_ref.remotes.origin
      repo_origin.push()

#-> /usr/sbin/httpd -V

Server version: Apache/2.4.6 (Red Hat Enterprise Linux)

Server built:   Sep 17 2015 09:06:30

Server's Module Magic Number: 20120211:24

Server loaded:  APR 1.4.8, APR-UTIL 1.5.2

Compiled using: APR 1.4.8, APR-UTIL 1.5.2

Architecture:   64-bit

Server MPM:     prefork

  threaded:     no

    forked:     yes (variable process count)

Server compiled with....

 -D APR_HAS_SENDFILE

 -D APR_HAS_MMAP

 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)

 -D APR_USE_SYSVSEM_SERIALIZE

 -D APR_USE_PTHREAD_SERIALIZE

 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT

 -D APR_HAS_OTHER_CHILD

 -D AP_HAVE_RELIABLE_PIPED_LOGS

 -D DYNAMIC_MODULE_LIMIT=256

 -D HTTPD_ROOT="/etc/httpd"

 -D SUEXEC_BIN="/usr/sbin/suexec"

 -D DEFAULT_PIDLOG="/run/httpd/httpd.pid"

 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"

 -D DEFAULT_ERRORLOG="logs/error_log"

 -D AP_TYPES_CONFIG_FILE="conf/mime.types"

 -D SERVER_CONFIG_FILE="conf/httpd.conf"

 

 

[Graham Dumpleton]

Few things to be aware of.

Your code under mod_wsgi by default runs as the Apache user. If you git operations rely on picking up SSH keys from your home account, it will not work. You would need to use daemon mode and set the user for the daemon mode account to be your user for it to find your SSH keys.

Next, ensure you are using daemon mode and that only that WSGI application is delegated to the daemon process group. Then ensure using main interpreter context by using:

    WSGIApplicationGroup %{GLOBAL}

in appropriate context, or use the:

    application-group=%{GLOBAL}

option to WSGIScriptAlias.

Finally on RHEL, it may have in place SELinux restrictions which are presenting the operation from succeeding.

Graham

-- 
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Joel Weinfeld]

Hi -

I set the  WSGIApplicationGroup  to the group ... it did not solve the problem

BTW ... our apache run as the owner of that dir ... and the env variables are set to point to the git settings

SELinux is disabled

#-> cat /etc/httpd/conf.d/microservices.conf
LoadModule wsgi_module /usr/lib64/httpd/modules/mod_wsgi.so
WSGIScriptAlias / /opt/xxx/cloud_automation/micro-services/flask.wsgi
#WSGIPythonPath /opt/xxx/cloud_automation/micro-services/:/usr/venv/lib/python3.6/site-packages
wSGIScriptReloading On
WSGIPassAuthorization On
WSGIRestrictStdout Off
WSGIRestrictStdin Off
WSGIRestrictSignal Off
WSGIApplicationGroup xxx
SetEnv GIT_CONFIG /home/xxx/.gitconfig
SetEnv GIT_PYTHON_TRACE full

....

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscribe@googlegroups.com.

To post to this group, send email to mod...@googlegroups.com.
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscribe@googlegroups.com.

[Graham Dumpleton]

On 11 Jun 2018, at 1:37 am, Joel Weinfeld <hobbit...@gmail.com> wrote:

Hi -

I set the  WSGIApplicationGroup  to the group ... it did not solve the problem

BTW ... our apache run as the owner of that dir ... and the env variables are set to point to the git settings

Not really a good idea to run the whole of Apache as a different user. Use mod_wsgi daemon mode and set up just that daemon process group to run as the different user, but only change the user for that even if it is essential.

SELinux is disabled

#-> cat /etc/httpd/conf.d/microservices.conf
LoadModule wsgi_module /usr/lib64/httpd/modules/mod_wsgi.so
WSGIScriptAlias / /opt/xxx/cloud_automation/micro-services/flask.wsgi
#WSGIPythonPath /opt/xxx/cloud_automation/micro-services/:/usr/venv/lib/python3.6/site-packages
wSGIScriptReloading On

WSGIScriptReloading is not needed. That is the default.

WSGIPassAuthorization On

Only needed if your WSGI application is handling HTTP basic authentication.

WSGIRestrictStdout Off
WSGIRestrictStdin Off

Neither of these is needed. They are the default.

WSGIRestrictSignal Off

You should not disable the restrictions on setting signal handlers unless you have an absolute need. This can interfere with operation of Apache and mod_wsgi. Setting your own signal handlers will also only work in a very specific use case when using Apache with multi process and single thread configuration.

WSGIApplicationGroup xxx

You should look at using daemon mode of mod_wsgi. Don't use embedded mode unless you have a very specific need that requires it.

SetEnv GIT_CONFIG /home/xxx/.gitconfig
SetEnv GIT_PYTHON_TRACE full

These will not set process environment variables in the processes if that is what you are expecting. This likely is why things aren't working.

You should use daemon mode. When you have daemon mode running, set the environment variables in os.environ at the start of your WSGI script file.

Graham

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.

[Joel Weinfeld]

Hi -

I added in flask.wsgi

            os.environ['GIT_CONFIG'] = "/home/xxx/.gitconfig"
            os.environ['GIT_PYTHON_TRACE'] = "full"
 

It still did not work ...

I ended up changing the git push to use pygit2 to do the push ... and it works ... there is something wrong with the way gitpython does a push under mod_wsgi

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscribe@googlegroups.com.

To post to this group, send email to mod...@googlegroups.com.
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Graham Dumpleton]

Comments in their documentation suggest that it would be a poor choice to use with mod_wsgi even if it did work. That is:

* https://github.com/gitpython-developers/GitPython#leakage-of-system-resources

GitPython is not suited for long-running processes (like daemons) as it tends to leak system resources. It was written in a time where destructors (as implemented in the __del__ method) still ran deterministically.

In case you still want to use it in such a context, you will want to search the codebase for __del__ implementations and call these yourself when you see fit.

Not sure if was gitpython, but have encountered previously case where someone was doing stuff with git under the covers and was getting problems with crashes due to really bad practices being done in __del__() methods of the Python package they were using which dealt with git. It may well have been gitpython if they are so dependent on __del__() methods being triggered immediately.

So if pygit2 works, maybe it is a better solution anyway.

Graham

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.