Webfaction mod_wsgi + Django

[Paul Royik]

I moved my hosting to Webfaction and problems began.

Webfaction reported that my memory consumption is too high (this wasn't an issue on previous hosting).

They don't tell you what page generated it or what objects were loaded in memory.

Support directed me to https://code.google.com/p/modwsgi/wiki/DebuggingTechniques, but I don't understand how to include that code into my project.

Please, help me. I really need to undestand what objects are loaded into memory.

[Graham Dumpleton]

What version of Apache and mod_wsgi does WebFaction use?

If they are still using Apache 2.2 and also an older mod_wsgi version, that is likely to be the cause.

Apache 2.2 has various issues in it with excessive memory usage that Apache 2.4 fixes.

Newer versions of mod_wsgi also provide workarounds to combat the Apache 2.2 memory issues, so if you are stuck with older Apache 2.2, then using latest mod_wsgi versions can still save the day.

Problem is that Linux distros that WebFaction uses are likely to use quite old versions of everything.

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Paul Royik]

They use Apache 2.2 and mod_wsgi 3.4

How can I upgrade them from command line.

Please guide me, as I'm very new to this. 

[Graham Dumpleton]

It has been a very long time since I have logged onto a WebFaction system so my memory of it is very slim.

There are also two approaches one can take to getting you running on a newer mod_wsgi version.

I therefore need you to do a title bit of home work for me so I know how WebFaction is currently setting up the system and can advise the best path forward.

The first thing we need to work out is whether the WebFaction Apache installation is a complete installation, or whether it is missing the header files and build tools required to even allow you to build mod_wsgi from source code. If they are missing, then there may not be a lot you can do.

So, can you see if you get anything when you run:

    which apxs

If this doesn't find where apxs is installed, we need to work out where the Apache being used may even be installed.

I have a vague recollection they have their own installation under /usr/local somewhere.

According to:

    http://docs.webfaction.com/software/django/config.html#django-restarting

which although Django specific is I believe not too much different even if using another framework, you should have a few admin scripts they generate to allow you to start and stop Apache.

Find therefore the equivalent for your system for:

    $HOME/webapps/django_app/apache2/bin/

What is in that directory:

    ls -las $HOME/webapps/django_app/apache2/bin/

There should be start, stop and restart scripts at least.

What is in the 'start' script itself.

Also what else is under the 'apache2' directory:

    ls -las $HOME/webapps/django_app/apache2

With that information, can then work out where to look next for what else I need to know.

Graham

[Paul Royik]

which apxs outputs:

"/usr/bin/which: no apxs in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/simamura/bin)"

in each aplication there is indeed folder apache2, so output to ls -las $HOME/webapps/django_app/apache2/bin/ is

total 1296

  4 drwxr-xr-x 2 simamura simamura   4096 Jan 23 12:42 .

  4 drwxr-xr-x 7 simamura simamura   4096 Jan 23 12:42 ..

632 -rwxr-xr-x 1 simamura simamura 644379 Oct  8  2013 httpd

644 -rwxr-xr-x 1 simamura simamura 658143 Oct  8  2013 httpd.worker

  4 -rwxr-xr-x 1 simamura simamura    126 Jan 23 12:42 restart

  4 -rwxr-xr-x 1 simamura simamura    207 Jan 23 12:42 start

  4 -rwxr-xr-x 1 simamura simamura    333 Jan 23 12:42 stop

Output of ls -las $HOME/webapps/django_app/apache2 is

total 28

4 drwxr-xr-x 7 simamura simamura 4096 Jan 23 12:42 .

4 drwxr-xr-x 6 simamura simamura 4096 Jan 23 19:15 ..

4 drwxr-xr-x 2 simamura simamura 4096 Jan 23 12:42 bin

4 drwxr-xr-x 2 simamura simamura 4096 Jan 23 12:42 conf

4 drwxr-xr-x 2 simamura simamura 4096 Oct  8  2013 lib

4 drwxr-xr-x 2 simamura simamura 4096 Jan 28 13:35 logs

4 drwxr-xr-x 2 simamura simamura 4096 Oct  8  2013 modules

Finally,  content of start script is 

#!/bin/bash

LD_LIBRARY_PATH=/home/simamura/webapps/django_app/apache2/lib /home/simamura/webapps/django_app/apache2/bin/httpd.worker -f /home/simamura/webapps/django_app/apache2/conf/httpd.conf -k start

Thank you.

[Graham Dumpleton]

Can you show me what is in:

    /home/simamura/webapps/django_app/apache2/conf/httpd.conf

and also show me what you get if you run:

    $HOME/webapps/django_app/apache2/bin/httpd.worker -V

Finally, since I recollect they stashed the original Apache under /usr/local, what do you get for:

    ls -las /usr/local

Graham

[Paul Royik]

Contents of httpd.conf

ServerRoot "/home/simamura/webapps/django_app/apache2"

LoadModule dir_module        modules/mod_dir.so

LoadModule env_module        modules/mod_env.so

LoadModule log_config_module modules/mod_log_config.so

LoadModule mime_module       modules/mod_mime.so

LoadModule rewrite_module    modules/mod_rewrite.so

LoadModule setenvif_module   modules/mod_setenvif.so

LoadModule wsgi_module       modules/mod_wsgi.so

LoadModule alias_module      modules/mod_alias.so

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

CustomLog /home/simamura/logs/user/access_django_app.log combined

ErrorLog /home/simamura/logs/user/error_django_app.log

KeepAlive Off

Listen 20241

MaxSpareThreads 3

MinSpareThreads 1

ServerLimit 1

SetEnvIf X-Forwarded-SSL on HTTPS=1

ThreadsPerChild 5

WSGIDaemonProcess django_app processes=2 threads=12 python-path=/home/simamura/webapps/django_app:/home/simamura/webapps/django_app/mathsite:/home/simamura/webapps/django_app/lib/python2.7

WSGIProcessGroup django_app

WSGIRestrictEmbedded On

WSGILazyInitialization On

WSGIScriptAlias / /home/simamura/webapps/django_app/proj/proj/wsgi.py

$HOME/webapps/django_app/apache2/bin/httpd.worker -V

Server version: Apache/2.2.25 (Unix)

Server built:   Sep  9 2013 06:59:27

Server's Module Magic Number: 20051115:33

Server loaded:  APR 1.3.9, APR-Util 1.3.9

Compiled using: APR 1.3.9, APR-Util 1.3.9

Architecture:   64-bit

Server MPM:     Worker

  threaded:     yes (fixed thread count)

    forked:     yes (variable process count)

Server compiled with....

 -D APACHE_MPM_DIR="server/mpm/worker"

 -D APR_HAS_SENDFILE

 -D APR_HAS_MMAP

 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)

 -D APR_USE_SYSVSEM_SERIALIZE

 -D APR_USE_PTHREAD_SERIALIZE

 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT

 -D APR_HAS_OTHER_CHILD

 -D AP_HAVE_RELIABLE_PIPED_LOGS

 -D DYNAMIC_MODULE_LIMIT=128

 -D HTTPD_ROOT="/usr/local/apache2"

 -D SUEXEC_BIN="/usr/local/apache2/bin/suexec"

 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"

 -D DEFAULT_ERRORLOG="logs/error_log"

 -D AP_TYPES_CONFIG_FILE="conf/mime.types"

 -D SERVER_CONFIG_FILE="conf/httpd.conf"

ls -las /usr/local

total 108

 4 drwxr-xr-x. 23 root   root    4096 Jun 10  2014 .

 4 drwxr-xr-x. 15 root   root    4096 Sep 14  2011 ..

 4 drwxr-xr-x   3 root   root    4096 Dec 12  2012 apache2-mpm-peruser

 4 drwxr-xr-x.  2 root   root    4096 Jan 28 18:37 bin

 4 drwxr-xr-x   3 root   root    4096 Dec  5  2010 burnintest

 4 drwxr-xr-x.  5 root   root    4096 Dec 30  2013 etc

 4 drwxr-xr-x.  2 root   root    4096 Sep 23  2011 games

 4 drwxr-xr-x. 22 root   root    4096 Oct 20 07:45 include

 4 drwxr-xr-x. 21 root   root    4096 Jan 28 18:37 lib

 4 drwxr-xr-x.  3 root   root    4096 Dec 12  2012 lib64

 4 drwxr-xr-x.  2 root   root    4096 Feb 20  2014 libexec

 4 drwxr-xr-x   7 root   root    4096 Jul 27  2014 maldetect

 4 drwxr-xr-x   3 root   root    4096 Dec 12  2012 man

 4 drwxr-xr-x   5 nagios nagios  4096 Jun 10  2014 nagios

 0 lrwxrwxrwx   1 root   root      14 Dec 12  2012 pgsql -> /usr/pgsql-9.1

 4 drwxr-xr-x   3 root   root    4096 Dec 12  2012 php

 4 drwxr-xr-x   2 root   root    4096 Dec 12  2012 pysqlite2-doc

 4 drwxr-xr-x   6 root   root    4096 Dec 12  2012 python3.3

 4 drwxr-xr-x   5 root   root    4096 Sep  1  2013 ruby1.8.7ee

 4 drwxr-xr-x.  2 root   root    4096 Jan 16 14:15 sbin

 4 drwxr-xr-x. 12 root   root    4096 Dec 30  2013 share

 4 drwxr-xr-x.  2 root   root    4096 Sep 23  2011 src

20 drwxr-xr-x   2 root   root   20480 Jan 29 03:00 users

 4 drwxr-xr-x   7 root   root    4096 Dec 12  2012 Zend

Thank you.

Paul.

[Graham Dumpleton]

WebFaction certainly looks to be making this very hard.

Lets try a few more places and if what I am looking for isn't there, I think you are out of luck as far as easy path goes.

So, what do you get for:

    ls -las /usr/local/bin

    ls -las /usr/local/sbin

    ls -las /usr/local/include

BTW, the Apache configuration you are using has problems as well. You are actually wasting memory on the daemon process configuration.

As is the maximum number of concurrent requests Apache itself can handle is 5. Your daemon processes are though setup to be able to handle 24. They can never do that though as Apache can only proxy at most 5 at any one time to the daemon processes.

You can halve your memory usage and likely see no difference in performance by using:

    WSGIDaemonProcess django_app threads=5

There are also issues with MinSpareThreads and MaxSpareThreads but Apache will correct that and result is no different.

I could suggest another tweak, but little point at the moment as other issues in Apache 2.2 nullify the usefulness of it anyway.

I really hope that isn't want WebFaction generates as a default.

BTW, what other Apache modules are in:

    $HOME/webapps/django_app/apache2/modules

Graham

[Paul Royik]

ls -las /usr/local/bin

total 136596

    4 drwxr-xr-x.  2 root root     4096 Jan 28 18:37 .

    4 drwxr-xr-x. 23 root root     4096 Jun 10  2014 ..

    0 lrwxrwxrwx   1 root root        8 Jul 24  2014 2to3 -> 2to3-3.3

    4 -rwxr-xr-x   1 root root      101 Jul 22  2014 2to3-3.3

    4 -rwxr-xr-x   1 root root      101 Oct  7 18:14 2to3-3.4

   12 -rwxr-xr-x   1 root root     9239 Sep 13  2011 amrnb-decoder

   12 -rwxr-xr-x   1 root root     9240 Sep 13  2011 amrnb-decoder-etsi

   12 -rwxr-xr-x   1 root root     9239 Sep 13  2011 amrnb-decoder-if2

   12 -rwxr-xr-x   1 root root     9963 Sep 13  2011 amrnb-encoder

   12 -rwxr-xr-x   1 root root     9884 Sep 13  2011 amrnb-encoder-etsi

   12 -rwxr-xr-x   1 root root     9884 Sep 13  2011 amrnb-encoder-etsi-vad2

   12 -rwxr-xr-x   1 root root     9899 Sep 13  2011 amrnb-encoder-if2

   12 -rwxr-xr-x   1 root root     9899 Sep 13  2011 amrnb-encoder-if2-vad2

   12 -rwxr-xr-x   1 root root     9963 Sep 13  2011 amrnb-encoder-vad2

   12 -rwxr-xr-x   1 root root     9485 Sep 13  2011 amrwb-decoder

   12 -rwxr-xr-x   1 root root     9357 Sep 13  2011 amrwb-decoder-if2

   12 -rwxr-xr-x   1 root root    10690 Sep 13  2011 amrwb-encoder

   12 -rwxr-xr-x   1 root root    10634 Sep 13  2011 amrwb-encoder-if2

    4 -rwxr-xr-x   1 root root      194 Dec  5  2010 burnin

    4 -rwxr-xr-x   1 root root      342 Apr 15  2014 ddmesg

    4 -rwxr-xr-x   1 root root      329 Feb  7  2014 easy_install

    4 -rwxr-xr-x   1 root root      337 Feb  7  2014 easy_install-2.4

    4 -rwxr-xr-x   1 root root      337 Feb  1  2014 easy_install-2.5

    4 -rwxr-xr-x   1 root root      331 Feb  1  2014 easy_install-2.7

    4 -rwxr-xr-x   1 root root      331 Feb  1  2014 easy_install-3.1

    4 -rwxr-xr-x   1 root root      331 Feb  1  2014 easy_install-3.2

    4 -rwxr-xr-x   1 root root      331 Feb  1  2014 easy_install-3.3

    4 -rwxr-xr-x   1 root root      241 Oct  7 18:14 easy_install-3.4

    0 lrwxrwxrwx   1 root root       22 Sep  2  2013 erb1.8.7ee -> ../ruby1.8.7ee/bin/erb

    8 -rwxr-xr-x   1 root root     4402 Jun 27  2013 erb1.9

    8 -rwxr-xr-x   1 root root     4402 Jun 27  2013 erb2.0

    8 -rwxr-xr-x   1 root root     4402 Aug  1 08:21 erb2.1

    4 -rwxr-xr-x   1 root root      675 Sep 14  2011 f2py2.4

    4 -rwxr-xr-x   1 root root      675 Sep 13  2011 f2py2.5

    4 -rwxr-xr-x   1 root root      675 Sep 14  2011 f2py2.7

  536 -rwxr-xr-x   1 root root   548459 Sep 13  2011 faac

   56 -rwxr-xr-x   1 root root    54347 Sep 13  2011 faad

 6112 -rwxr-xr-x   1 root root  6255040 Sep 13  2011 ffmpeg

 6088 -rwxr-xr-x   1 root root  6230368 Sep 13  2011 ffserver

    0 lrwxrwxrwx   1 root root       22 Sep  2  2013 gem1.8.7ee -> ../ruby1.8.7ee/bin/gem

    4 -rwxr-xr-x   1 root root      551 Jun 27  2013 gem1.9

    4 -rwxr-xr-x   1 root root      551 Jun 27  2013 gem2.0

    4 -rwxr-xr-x   1 root root      551 Aug  1 08:21 gem2.1

    4 -rwxr-xr-x   1 root root       99 Dec 14 05:15 idle

    4 -rwxr-xr-x   1 root root       99 May 18  2012 idle3

    4 -rwxr-xr-x   1 root root       99 May 16  2013 idle3.2

    4 -rwxr-xr-x   1 root root       99 Jul 22  2014 idle3.3

    4 -rwxr-xr-x   1 root root       99 Oct  7 18:14 idle3.4

    0 lrwxrwxrwx   1 root root       22 Sep  2  2013 irb1.8.7ee -> ../ruby1.8.7ee/bin/irb

    4 -rwxr-xr-x   1 root root      323 Jun 27  2013 irb1.9

    4 -rwxr-xr-x   1 root root      323 Jun 27  2013 irb2.0

    4 -rwxr-xr-x   1 root root      195 Aug  1 08:21 irb2.1

  564 -rwxr-xr-x   1 root root   577109 Sep 13  2011 lame

    4 -rwxr-xr-x   1 root root      354 Dec 12  2012 mongrel_rails

  100 -rwxr-xr-x   1 root root   101328 Aug 25  2013 nethogs

    4 -rwxr-xr-x   1 root root      799 Aug  3  2013 pear

    4 -rwxr-xr-x   1 root root      202 Sep 12  2011 pear53

    4 -rwxr-xr-x   1 root root      202 Mar 16  2012 pear54

    4 -rwxr-xr-x   1 root root      202 Dec 15  2013 pear55

    4 -rwxr-xr-x   1 root root      820 Aug  3  2013 peardev

    4 -rwxr-xr-x   1 root root      226 Sep 12  2011 peardev53

    4 -rwxr-xr-x   1 root root      226 Mar 16  2012 peardev54

    4 -rwxr-xr-x   1 root root      226 Dec 15  2013 peardev55

    4 -rwxr-xr-x   1 root root      736 Aug  3  2013 pecl

    4 -rwxr-xr-x   1 root root      208 Sep 12  2011 pecl53

    4 -rwxr-xr-x   1 root root      208 Mar 16  2012 pecl54

    4 -rwxr-xr-x   1 root root      208 Dec 15  2013 pecl55

    0 lrwxrwxrwx   1 root root       11 Aug 12  2013 phar53 -> phar53.phar

   16 -rwxr-xr-x   1 root root    14830 Aug  7  2013 phar53.phar

    0 lrwxrwxrwx   1 root root       11 Aug 12  2013 phar54 -> phar54.phar

   16 -rwxr-xr-x   1 root root    14825 Aug  2  2013 phar54.phar

    0 lrwxrwxrwx   1 root root       11 Dec 30  2013 phar55 -> phar55.phar

   16 -rwxr-xr-x   1 root root    14825 Dec 15  2013 phar55.phar

 8064 -rwxr-xr-x   1 root root  8257519 Aug  3  2013 php

11488 -rwxr-xr-x   1 root root 11761057 Aug  7  2013 php53

    4 -rwxr-xr-x   1 root root     3449 Aug  7  2013 php53-config

11460 -rwxr-xr-x   1 root root 11731157 Aug  2  2013 php54

    4 -rwxr-xr-x   1 root root     3402 Aug  2  2013 php54-config

12028 -rwxr-xr-x   1 root root 12314393 Dec 15  2013 php55

    4 -rwxr-xr-x   1 root root     3390 Dec 15  2013 php55-config

 8080 -rwxr-xr-x   1 root root  8272965 Aug  3  2013 php-cgi

    4 -rwxr-xr-x   1 root root     3242 Aug  3  2013 php-config

    8 -rwxr-xr-x   1 root root     4502 Aug  3  2013 phpize

    8 -rwxr-xr-x   1 root root     4506 Aug  7  2013 phpize53

    8 -rwxr-xr-x   1 root root     4535 Aug  2  2013 phpize54

    8 -rwxr-xr-x   1 root root     4526 Dec 15  2013 phpize55

    4 -rwxr-xr-x   1 root root     2322 Sep 13  2011 pilconvert.py

   16 -rwxr-xr-x   1 root root    15358 Sep 13  2011 pildriver.py

    4 -rwxr-xr-x   1 root root     2554 Sep 13  2011 pilfile.py

    4 -rwxrwxr-x   1 root root     1007 Dec  3  2006 pilfont.py

    4 -rwxr-xr-x   1 root root     2346 Sep 13  2011 pilprint.py

    4 -rwxr-xr-x   1 root root      213 Oct  7 18:14 pip3.4

    4 -rwxr-xr-x   1 root root       84 Dec 14 05:15 pydoc

    4 -rwxr-xr-x   1 root root       84 May 18  2012 pydoc3

    4 -rwxr-xr-x   1 root root       84 May 16  2013 pydoc3.2

    4 -rwxr-xr-x   1 root root       84 Jul 22  2014 pydoc3.3

    4 -rwxr-xr-x   1 root root       84 Oct  7 18:14 pydoc3.4

    0 lrwxrwxrwx   1 root root       15 Dec 12  2012 python -> /usr/bin/python

    8 -rwxr-xr-x   1 root root     7020 Sep  8  2011 python2.4

    8 -rwxr-xr-x   1 root root     6956 Sep 13  2011 python2.5

    4 -rwxr-xr-x   1 root root     1424 Sep 13  2011 python2.5-config

    0 lrwxrwxrwx   1 root root       18 Dec 12  2012 python2.6 -> /usr/bin/python2.6

    8 -rwxr-xr-x   1 root root     6956 Dec 14 05:15 python2.7

    4 -rwxr-xr-x   1 root root     1687 Dec 14 05:15 python2.7-config

    0 lrwxrwxrwx   1 root root        9 Jul 24  2014 python3 -> python3.3

   12 -rwxr-xr-x   1 root root     8342 Mar 22  2012 python3.0

    4 -rwxr-xr-x   1 root root     1401 Mar 22  2012 python3.0-config

   12 -rwxr-xr-x   1 root root     9690 May 18  2012 python3.1

    4 -rwxr-xr-x   1 root root     1401 May 18  2012 python3.1-config

   12 -rwxr-xr-x   2 root root     9319 May 16  2013 python3.2

   12 -rwxr-xr-x   2 root root     9319 May 16  2013 python3.2m

    4 -rwxr-xr-x   1 root root     1884 May 16  2013 python3.2m-config

   12 -rwxr-xr-x   2 root root     9271 Jul 22  2014 python3.3

    0 lrwxrwxrwx   1 root root       17 Jul 24  2014 python3.3-config -> python3.3m-config

   12 -rwxr-xr-x   2 root root     9271 Jul 22  2014 python3.3m

    4 -rwxr-xr-x   1 root root     1978 Jul 22  2014 python3.3m-config

   12 -rwxr-xr-x   2 root root     8558 Oct  7 18:14 python3.4

    0 lrwxrwxrwx   1 root root       17 Oct 10 17:16 python3.4-config -> python3.4m-config

   12 -rwxr-xr-x   2 root root     8558 Oct  7 18:14 python3.4m

    4 -rwxr-xr-x   1 root root     3047 Oct  7 18:14 python3.4m-config

    0 lrwxrwxrwx   1 root root       16 Jul 24  2014 python3-config -> python3.3-config

    4 -rwxr-xr-x   1 root root      236 Jul 22  2014 pyvenv-3.3

    4 -rwxr-xr-x   1 root root      236 Oct  7 18:14 pyvenv-3.4

    4 -rwxr-xr-x   1 root root     1236 Jun 27  2013 rake1.9

    4 -rwxr-xr-x   1 root root     1325 Jun 27  2013 rake2.0

    4 -rwxr-xr-x   1 root root     1249 Aug  1 08:21 rake2.1

    0 lrwxrwxrwx   1 root root       23 Sep  2  2013 rdoc1.8.7ee -> ../ruby1.8.7ee/bin/rdoc

    4 -rwxr-xr-x   1 root root      793 Jun 27  2013 rdoc1.9

    4 -rwxr-xr-x   1 root root      943 Jun 27  2013 rdoc2.0

    4 -rwxr-xr-x   1 root root      943 Aug  1 08:21 rdoc2.1

    0 lrwxrwxrwx   1 root root       30 Sep  2  2013 ree-version1.8.7ee -> ../ruby1.8.7ee/bin/ree-version

    0 lrwxrwxrwx   1 root root       21 Sep  2  2013 ri1.8.7ee -> ../ruby1.8.7ee/bin/ri

    4 -rwxr-xr-x   1 root root      193 Jun 27  2013 ri1.9

    4 -rwxr-xr-x   1 root root      193 Jun 27  2013 ri2.0

    4 -rwxr-xr-x   1 root root      193 Aug  1 08:21 ri2.1

    4 -rwxr-xr-x   1 root root      602 Sep 14  2011 rst2html.py

    4 -rwxr-xr-x   1 root root      799 Sep 14  2011 rst2latex.py

    4 -rwxr-xr-x   1 root root      608 Sep 14  2011 rst2man.py

    4 -rwxr-xr-x   1 root root      735 Sep 14  2011 rst2newlatex.py

    4 -rwxr-xr-x   1 root root     1706 Sep 14  2011 rst2odt_prepstyles.py

    4 -rwxr-xr-x   1 root root      772 Sep 14  2011 rst2odt.py

    4 -rwxr-xr-x   1 root root      609 Sep 14  2011 rst2pseudoxml.py

    4 -rwxr-xr-x   1 root root      645 Sep 14  2011 rst2s5.py

    4 -rwxr-xr-x   1 root root      610 Sep 14  2011 rst2xml.py

    4 -rwxr-xr-x   1 root root      678 Sep 14  2011 rstpep2html.py

    0 lrwxrwxrwx   1 root root       13 Dec 12  2012 ruby -> /usr/bin/ruby

    0 lrwxrwxrwx   1 root root       13 Dec 12  2012 ruby1.8.7 -> /usr/bin/ruby

    0 lrwxrwxrwx   1 root root       23 Sep  2  2013 ruby1.8.7ee -> ../ruby1.8.7ee/bin/ruby

    8 -rwxr-xr-x   1 root root     7725 Jun 27  2013 ruby1.9

    8 -rwxr-xr-x   1 root root     7725 Jun 27  2013 ruby2.0

    8 -rwxr-xr-x   1 root root     7677 Aug  1 08:21 ruby2.1

   20 -rwxr-xr-x   1 root root    18547 Dec 14 05:15 smtpd.py

    0 lrwxrwxrwx   1 root root       25 Sep  2  2013 testrb1.8.7ee -> ../ruby1.8.7ee/bin/testrb

    4 -rwxr-xr-x   1 root root      303 Jun 27  2013 testrb1.9

    4 -rwxr-xr-x   1 root root       83 Jun 27  2013 testrb2.0

    4 -rwxr-xr-x   1 root root       83 Aug  1 08:21 testrb2.1

35484 -rwxr-xr-x   1 root root 36334480 Jun 26  2014 wkhtmltoimage

35548 -rwxr-xr-x   1 root root 36401072 Jun 26  2014 wkhtmltopdf

   92 -rwxr-xr-x   1 root root    92983 Oct  2  2012 zabbix_get

  112 -rwxr-xr-x   1 root root   111608 Oct  2  2012 zabbix_sender

ls -las /usr/local/sbin

total 632

  4 drwxr-xr-x.  2 root root   4096 Jan 16 14:15 .

  4 drwxr-xr-x. 23 root root   4096 Jun 10  2014 ..

  4 -rwxrwxr-x   1 root root   1218 May  7  2013 adminbot

 12 -rwx------   1 root root  10827 Jan  8 10:08 check_admin_logins

  4 -r-x------   1 root root   2147 Sep 19 20:28 check_apache

  8 -r-x------   1 root root   4689 Oct  6 12:46 check_filesystem

  4 -r-x------   1 root root   1951 Dec 26 14:58 check_monitor

  8 -rwx------   1 root root   5651 Oct  3 16:25 check_mysql

  4 -r-x------   1 root root   2132 Sep 19 20:11 check_partitions

 24 -r-x------   1 root root  20531 Dec 18 13:16 check_raid

  4 -r-x------   1 root root   1608 Sep 19 20:28 check_regenerate_webserver

  4 -r-x------   1 root root   2453 Dec 12  2012 check_rpm

  8 -r-x------   1 root root   6042 Sep 19 20:11 check_smart

  8 -r-x------   1 root root   7252 Sep 19 20:28 check_traffic

  4 -r-x------   1 root root   1190 Jul  3  2014 check_updates

  4 -r-x------   1 root root     30 Aug 26 12:06 count-apache

  4 -r-x------   1 root root   1242 Mar 10  2014 iptraf_report

 12 -r-x------   1 root root  11813 Jun  6  2014 iptraf_report_summarizer

  4 -r-x------   1 root root   1744 Aug 14 16:50 maldetect_update

  4 -r-xr-xr-x   1 root root   1373 Dec 18 17:46 mod_security_client.py

  4 -r-x------   1 root root   3081 Dec 15 16:47 mod_security_remove.py

  4 -r-x------   1 root root   2755 Dec 18 17:46 mod_security_server.py

  4 -r-x------   1 root root    216 Mar  1  2013 speedtest-disks

  4 -r-x------   1 root root   1446 Oct 11  2013 speedtest-mysql

  4 -r-x------   1 root root   2559 Nov 17 20:24 verify_rpms

224 -rwxr-xr-x   1 root root 227320 Oct  2  2012 zabbix_agent

256 -rwxr-xr-x   1 root root 262117 Oct  2  2012 zabbix_agentd

 ls -las /usr/local/include

total 132

 4 drwxr-xr-x. 22 root root  4096 Oct 20 07:45 .

 4 drwxr-xr-x. 23 root root  4096 Jun 10  2014 ..

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 amrnb

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 amrwb

 4 -rw-r--r--   1 root root  3059 Sep 13  2011 faaccfg.h

 4 -rw-r--r--   1 root root  2363 Sep 13  2011 faac.h

 4 -rw-r--r--   1 root root  1461 Sep 13  2011 faad.h

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 lame

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 libavcodec

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 libavdevice

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 libavformat

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 libavutil

 8 -rw-r--r--   1 root root  5542 Sep 13  2011 mp4ff.h

12 -rw-r--r--   1 root root 11106 Sep 13  2011 mp4ffint.h

12 -rw-r--r--   1 root root  8463 Sep 13  2011 neaacdec.h

 4 drwxr-xr-x   8 root root  4096 Aug  3  2013 php

 4 drwxr-xr-x   7 root root  4096 Aug  7  2013 php53

 4 drwxr-xr-x   8 root root  4096 Aug  2  2013 php54

 4 drwxr-xr-x   8 root root  4096 Dec 30  2013 php55

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 python2.4

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 python2.5

 4 drwxr-xr-x   2 root root  4096 Jan 28 18:37 python2.7

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 python3.0

 4 drwxr-xr-x   2 root root  4096 Dec 12  2012 python3.1

 4 drwxr-xr-x   2 root root  4096 May 27  2013 python3.2m

 4 drwxr-xr-x   2 root root  4096 Jul 24  2014 python3.3m

 4 drwxr-xr-x   2 root root  4096 Oct 10 17:16 python3.4m

 4 drwxr-xr-x   2 root root  4096 Oct 20 07:45 wkhtmltox

$HOME/webapps/django_app/apache2/modules

total 1480

  4 drwxr-xr-x 2 simamura simamura   4096 Oct  8  2013 .

  4 drwxr-xr-x 7 simamura simamura   4096 Jan 23 12:42 ..

 12 -rwxr-xr-x 1 simamura simamura  10590 Oct  8  2013 mod_actions.so

 20 -rwxr-xr-x 1 simamura simamura  17213 Oct  8  2013 mod_alias.so

 12 -rwxr-xr-x 1 simamura simamura   9422 Oct  8  2013 mod_asis.so

 12 -rwxr-xr-x 1 simamura simamura  11966 Oct  8  2013 mod_auth_basic.so

 40 -rwxr-xr-x 1 simamura simamura  38828 Oct  8  2013 mod_auth_digest.so

 12 -rwxr-xr-x 1 simamura simamura   9563 Oct  8  2013 mod_authn_anon.so

 16 -rwxr-xr-x 1 simamura simamura  13175 Oct  8  2013 mod_authn_dbd.so

 12 -rwxr-xr-x 1 simamura simamura   9904 Oct  8  2013 mod_authn_dbm.so

 12 -rwxr-xr-x 1 simamura simamura   8268 Oct  8  2013 mod_authn_default.so

 12 -rwxr-xr-x 1 simamura simamura   9795 Oct  8  2013 mod_authn_file.so

 12 -rwxr-xr-x 1 simamura simamura  12143 Oct  8  2013 mod_authz_dbm.so

  8 -rwxr-xr-x 1 simamura simamura   7978 Oct  8  2013 mod_authz_default.so

 16 -rwxr-xr-x 1 simamura simamura  12676 Oct  8  2013 mod_authz_groupfile.so

 16 -rwxr-xr-x 1 simamura simamura  12297 Oct  8  2013 mod_authz_host.so

 12 -rwxr-xr-x 1 simamura simamura  10137 Oct  8  2013 mod_authz_owner.so

 12 -rwxr-xr-x 1 simamura simamura   9355 Oct  8  2013 mod_authz_user.so

 44 -rwxr-xr-x 1 simamura simamura  44517 Oct  8  2013 mod_autoindex.so

 12 -rwxr-xr-x 1 simamura simamura  12161 Oct  8  2013 mod_cern_meta.so

 36 -rwxr-xr-x 1 simamura simamura  34956 Oct  8  2013 mod_cgi.so

 68 -rwxr-xr-x 1 simamura simamura  67668 Oct  8  2013 mod_dav_fs.so

124 -rwxr-xr-x 1 simamura simamura 124246 Oct  8  2013 mod_dav.so

 24 -rwxr-xr-x 1 simamura simamura  24292 Oct  8  2013 mod_dbd.so

 32 -rwxr-xr-x 1 simamura simamura  32604 Oct  8  2013 mod_deflate.so

 12 -rwxr-xr-x 1 simamura simamura  12080 Oct  8  2013 mod_dir.so

 16 -rwxr-xr-x 1 simamura simamura  12679 Oct  8  2013 mod_dumpio.so

 12 -rwxr-xr-x 1 simamura simamura   9902 Oct  8  2013 mod_env.so

 16 -rwxr-xr-x 1 simamura simamura  14662 Oct  8  2013 mod_expires.so

 28 -rwxr-xr-x 1 simamura simamura  27676 Oct  8  2013 mod_ext_filter.so

 20 -rwxr-xr-x 1 simamura simamura  20459 Oct  8  2013 mod_filter.so

 28 -rwxr-xr-x 1 simamura simamura  24924 Oct  8  2013 mod_headers.so

 12 -rwxr-xr-x 1 simamura simamura  12056 Oct  8  2013 mod_ident.so

 24 -rwxr-xr-x 1 simamura simamura  20761 Oct  8  2013 mod_imagemap.so

 64 -rwxr-xr-x 1 simamura simamura  63586 Oct  8  2013 mod_include.so

 28 -rwxr-xr-x 1 simamura simamura  26052 Oct  8  2013 mod_info.so

 36 -rwxr-xr-x 1 simamura simamura  35488 Oct  8  2013 mod_log_config.so

 16 -rwxr-xr-x 1 simamura simamura  13423 Oct  8  2013 mod_log_forensic.so

 12 -rwxr-xr-x 1 simamura simamura  10706 Oct  8  2013 mod_logio.so

 36 -rwxr-xr-x 1 simamura simamura  33694 Oct  8  2013 mod_mime_magic.so

 24 -rwxr-xr-x 1 simamura simamura  24319 Oct  8  2013 mod_mime.so

 44 -rwxr-xr-x 1 simamura simamura  43048 Oct  8  2013 mod_negotiation.so

 20 -rwxr-xr-x 1 simamura simamura  16503 Oct  8  2013 mod_reqtimeout.so

 84 -rwxr-xr-x 1 simamura simamura  84448 Oct  8  2013 mod_rewrite.so

 16 -rwxr-xr-x 1 simamura simamura  15951 Oct  8  2013 mod_setenvif.so

 16 -rwxr-xr-x 1 simamura simamura  15679 Oct  8  2013 mod_speling.so

 28 -rwxr-xr-x 1 simamura simamura  25338 Oct  8  2013 mod_status.so

 20 -rwxr-xr-x 1 simamura simamura  17319 Oct  8  2013 mod_substitute.so

 12 -rwxr-xr-x 1 simamura simamura  11276 Oct  8  2013 mod_unique_id.so

 12 -rwxr-xr-x 1 simamura simamura  11898 Oct  8  2013 mod_userdir.so

 16 -rwxr-xr-x 1 simamura simamura  15230 Oct  8  2013 mod_usertrack.so

 12 -rwxr-xr-x 1 simamura simamura  10311 Oct  8  2013 mod_version.so

 16 -rwxr-xr-x 1 simamura simamura  13201 Oct  8  2013 mod_vhost_alias.so

216 -rwxr-xr-x 1 simamura simamura 220299 Oct  8  2013 mod_wsgi.so

[Graham Dumpleton]

What you need is missing.

In:

    https://twitter.com/GrahamDumpleton/status/560759616867295232

WebFaction has basically acknowledged it is impossible without building your own Apache.

The page they refer to is quite out of date though in respect of the versions it uses.

I will not get to it until tomorrow, but I can provide you a buildout description file for compiling an up to date version of Apache if you really want to go down that path.

With that installed, you can either then build mod_wsgi from source code in traditional way, or instead use mod_wsgi-express, which makes it all somewhat easier and provides you a better curated Apache configuration as well.

If interested in what mod_wsgi-express is, have a look at:

    https://pypi.python.org/pypi/mod_wsgi

Graham

[Paul Royik]

I want advice what to do

If you think that proposed path is good, then let' go for it.

[Graham Dumpleton]

On 30/01/2015, at 3:33 AM, Paul Royik <distan...@gmail.com> wrote:

> I want advice what to do
>
> If you think that proposed path is good, then let' go for it.

As first step, log into your WebFaction account.

I don't know how WebFaction sets you up with a writable Python installation, but if you are using a Python virtual environment ensure that it is activated as you would if you are installing any other Python packages.

Then run:

MOD_WSGI_REQUIRE_APACHE=true pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip

This will take a while.

What it will do is the following.

1. Download latest apr, apr-util and httpd source code.
2. Build Apache httpd from the source code and installing it as a special package into your Python installation.
3. Build mod_wsgi against the installed Apache installation.

The end result of this will be the command 'mod_wsgi-express'. When it is run, it will run up Apache using the version installed by the package. It will even generate the Apache configuration for you.

Anyway, first step is to see whether the installation works okay. We can then go through what options you will need to use with mod_wsgi-express to give you the same result as far as running up your Python web application. We will also need to work out where is the correct place to integrate it into WebFaction to get this server started up instead of the default Apache one they give you.

On this later point, my recollection is that they use cron to ensure your Apache is running, so can you give the output from running:

crontab -l

Thanks.

Graham

[Paul Royik]

I don't use virtualenv.

Do I need to install it?

Also,

crontab -l  should be run after MOD_WSGI_REQUIRE_APACHE=true pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip  ?

[Graham Dumpleton]

On 30/01/2015, at 11:08 PM, Paul Royik <distan...@gmail.com> wrote:

I don't use virtualenv.

Do I need to install it?

No. Sounds like WebFaction already sets up something for you else you wouldn't be able to install any Python packages.

Also,

crontab -l  should be run after MOD_WSGI_REQUIRE_APACHE=true pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip  ?

The running of 'crontab -l' is completely separate thing and nothing to do with running the pip command. It is just verifying for me what WebFaction has set up for you in the crontab so I know what to change it to when we get to the next phase.

First step is just to run that pip command and see if it all installs properly.

Graham

[Paul Royik]

After running command I've got the following:

    100% |################################| 675kB 7.1MB/s

    sh: ./configure: Permission denied

    Downloading http://www.us.apache.org/dist/apr/apr-1.5.1.tar.gz

    Downloading http://www.us.apache.org/dist/apr/apr-util-1.5.4.tar.gz

    Downloading http://www.us.apache.org/dist/httpd/httpd-2.4.12.tar.gz

    Extracting apr-1.5.1.tar.gz

    Extracting apr-util-1.5.4.tar.gz

    Extracting httpd-2.4.12.tar.gz

    Traceback (most recent call last):

      File "<string>", line 20, in <module>

      File "/tmp/pip-Cs5fhp-build/setup.py", line 86, in <module>

        raise RuntimeError('Failed to build APR.')

    RuntimeError: Failed to build APR.

    Complete output from command python setup.py egg_info:

    sh: ./configure: Permission denied

    Downloading http://www.us.apache.org/dist/apr/apr-1.5.1.tar.gz

    Downloading http://www.us.apache.org/dist/apr/apr-util-1.5.4.tar.gz

    Downloading http://www.us.apache.org/dist/httpd/httpd-2.4.12.tar.gz

    Extracting apr-1.5.1.tar.gz

    Extracting apr-util-1.5.4.tar.gz

    Extracting httpd-2.4.12.tar.gz

    Traceback (most recent call last):

      File "<string>", line 20, in <module>

      File "/tmp/pip-Cs5fhp-build/setup.py", line 86, in <module>

        raise RuntimeError('Failed to build APR.')

    RuntimeError: Failed to build APR.

    ----------------------------------------

    Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-Cs5f                                                                                                                     hp-build

[Graham Dumpleton]

What do you get if you run:

    umask

That is the very last thing would expect to see and would suggest your shell environment has a strange permissions setup.

Graham

[Graham Dumpleton]

And what do you get for:

    python -v

That is, after the full python version.

Graham

[Paul Royik]

umask gives

0002

pyhton -v

# installing zipimport hook

import zipimport # builtin

# installed zipimport hook

# /usr/lib64/python2.6/site.pyc matches /usr/lib64/python2.6/site.py

import site # precompiled from /usr/lib64/python2.6/site.pyc

# /usr/lib64/python2.6/os.pyc matches /usr/lib64/python2.6/os.py

import os # precompiled from /usr/lib64/python2.6/os.pyc

import errno # builtin

import posix # builtin

# /usr/lib64/python2.6/posixpath.pyc matches /usr/lib64/python2.6/posixpath.py

import posixpath # precompiled from /usr/lib64/python2.6/posixpath.pyc

# /usr/lib64/python2.6/stat.pyc matches /usr/lib64/python2.6/stat.py

import stat # precompiled from /usr/lib64/python2.6/stat.pyc

# /usr/lib64/python2.6/genericpath.pyc matches /usr/lib64/python2.6/genericpath.py

import genericpath # precompiled from /usr/lib64/python2.6/genericpath.pyc

# /usr/lib64/python2.6/warnings.pyc matches /usr/lib64/python2.6/warnings.py

import warnings # precompiled from /usr/lib64/python2.6/warnings.pyc

# /usr/lib64/python2.6/linecache.pyc matches /usr/lib64/python2.6/linecache.py

import linecache # precompiled from /usr/lib64/python2.6/linecache.pyc

# /usr/lib64/python2.6/types.pyc matches /usr/lib64/python2.6/types.py

import types # precompiled from /usr/lib64/python2.6/types.pyc

# /usr/lib64/python2.6/UserDict.pyc matches /usr/lib64/python2.6/UserDict.py

import UserDict # precompiled from /usr/lib64/python2.6/UserDict.pyc

# /usr/lib64/python2.6/_abcoll.pyc matches /usr/lib64/python2.6/_abcoll.py

import _abcoll # precompiled from /usr/lib64/python2.6/_abcoll.pyc

# /usr/lib64/python2.6/abc.pyc matches /usr/lib64/python2.6/abc.py

import abc # precompiled from /usr/lib64/python2.6/abc.pyc

# /usr/lib64/python2.6/copy_reg.pyc matches /usr/lib64/python2.6/copy_reg.py

import copy_reg # precompiled from /usr/lib64/python2.6/copy_reg.pyc

# /usr/lib64/python2.6/sitecustomize.pyc matches /usr/lib64/python2.6/sitecustomize.py

import sitecustomize # precompiled from /usr/lib64/python2.6/sitecustomize.pyc

import encodings # directory /usr/lib64/python2.6/encodings

# /usr/lib64/python2.6/encodings/__init__.pyc matches /usr/lib64/python2.6/encodings/__init__.py

import encodings # precompiled from /usr/lib64/python2.6/encodings/__init__.pyc

# /usr/lib64/python2.6/codecs.pyc matches /usr/lib64/python2.6/codecs.py

import codecs # precompiled from /usr/lib64/python2.6/codecs.pyc

import _codecs # builtin

# /usr/lib64/python2.6/encodings/aliases.pyc matches /usr/lib64/python2.6/encodings/aliases.py

import encodings.aliases # precompiled from /usr/lib64/python2.6/encodings/aliases.pyc

# /usr/lib64/python2.6/encodings/utf_8.pyc matches /usr/lib64/python2.6/encodings/utf_8.py

import encodings.utf_8 # precompiled from /usr/lib64/python2.6/encodings/utf_8.pyc

Python 2.6.6 (r266:84292, Jan 22 2014, 09:42:36)

[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2

Type "help", "copyright", "credits" or "license" for more information.

dlopen("/usr/lib64/python2.6/lib-dynload/readline.so", 2);

import readline # dynamically loaded from /usr/lib64/python2.6/lib-dynload/readline.so

[Graham Dumpleton]

Okay, the reason it failed was because it was using Python 2.6. The tar file module in older Python versions didn't honour umask setting.

The issue now is what Python version do you want to be using?

Your Apache configuration had:

WSGIDaemonProcess django_app processes=2 threads=12 python-path=/home/simamura/webapps/django_app:/home/simamura/webapps/django_app/mathsite:/home/simamura/webapps/django_app/lib/python2.7

This indicates are probably trying to use Python 2.7 and not 2.6.

What command have you been using up till now to install Python packages? The default Python is 2.6 and not 2.7.

Were you explicitly using pip2.7 or python2.7 on package setup,py files?

Just to be absolute sure about what Python version you have been using, what do you get for:

ldd /home/simamura/webapps/django_app/apache2/modules/mod_wsgi.so

Anyway, I have decided to change the approach I am using to handle installing Apache and make it a package distinct from mod_wsgi package itself. I therefore need to do a bit more work for that.

Please do give me answers to the questions above though.

Graham

[Paul Royik]

I already noticed that in terminal Python 2.6 is used. I asked support how to change to installed 2.7 version, but did not get answer.

Were you explicitly using pip2.7 or python2.7 on package setup,py files? 

I used only pip, not pip 2.7

ldd /home/simamura/webapps/django_app/apache2/modules/mod_wsgi.so

        linux-vdso.so.1 =>  (0x00007fff1e17a000)

        libpython2.7.so.1.0 => /usr/local/lib/libpython2.7.so.1.0 (0x00007f8f14057000)

        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8f13e3a000)

        libdl.so.2 => /lib64/libdl.so.2 (0x00007f8f13c35000)

        libutil.so.1 => /lib64/libutil.so.1 (0x00007f8f13a32000)

        libm.so.6 => /lib64/libm.so.6 (0x00007f8f137ae000)

        libc.so.6 => /lib64/libc.so.6 (0x00007f8f13419000)

        /lib64/ld-linux-x86-64.so.2 (0x00007f8f14679000)

[Graham Dumpleton]

On 31/01/2015, at 8:43 PM, Paul Royik <distan...@gmail.com> wrote:

> I already noticed that in terminal Python 2.6 is used. I asked support how to change to installed 2.7 version, but did not get answer.
> Were you explicitly using pip2.7 or python2.7 on package setup,py files?
> I used only pip, not pip 2.7

Lets see if we can sort this out. WebFaction's setup for Python is quite messy based on their documentation. They have also modified their Python installations to do special things.

The page where they describe things is:

http://docs.webfaction.com/software/python.html

You ideally want to be setup with pip if you haven't already. This is described in:

http://docs.webfaction.com/software/python.html#installing-packages-with-pip

So what you want to do is:

mkdir -p $HOME/lib/python2.7

easy_install-2.7 pip

Whenever you need to know install packages you should always run 'pip2.7' and NOT just 'pip'. You should also provide the --user option to the 'install' command for pip.

As explained in:

http://docs.webfaction.com/software/python.html#creating-a-python-alias

you can create shell script aliases for 'python' and 'pip' to refer to the correct versions:

alias python=python2.7
alias pip=pip2.7

Now, with at least 'pip2.7' installed, run:

pip2.7 install --user mod_wsgi-httpd

This will take a while to run. This is going to compile and install Apache 2.4.12. This is now done by a separate package that is on PyPi.

If that appears to succeed, try running:

mod_wsgi-apxs -q INCLUDEDIR

If that runs we are in good shape.

If so far so good, then run:

pip2.7 install --user mod_wsgi

This will compile mod_wsgi package itself and install mod_wsgi-express.

If that looks to work, then run:

mod-wsgi-express

it should output a usage message.

Let me know when you get that far or at what point something doesn't work.

Graham

[Paul Royik]

It appears that I already installed pip 2.7 (because pip 2.7 ran without errors).

But  commmand  pip2.7 install --user mod_wsgi-httpd  gives error:

Downloading http://downloads.sourceforge.net/project/pcre/pcre/8.36/pcre-8.36.tar.gz

    Traceback (most recent call last):

      File "<string>", line 20, in <module>

      File "/tmp/pip-build-DnLt91/mod-wsgi-httpd/setup.py", line 77, in <module>

        raise RuntimeError('Failed to build APR.')

    RuntimeError: Failed to build APR.

    ----------------------------------------

    Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-DnLt91/mod-wsgi-httpd

[Graham Dumpleton]

What is in:

    $HOME/.pip/pip.log

The pip command can be a right pain in not showing all the output properly. The log file may have more information in it that I need for working out why it failed to build.

Graham

[Paul Royik]

There is no such file.

I looked over hidden files, but still without success.

[Graham Dumpleton]

On 02/02/2015, at 9:08 PM, Paul Royik <distan...@gmail.com> wrote:

> There is no such file.
> I looked over hidden files, but still without success.

What do you get when you run:

pip2.7 --version

[Paul Royik]

pip 6.0.6 from /home/username/lib/python2.7/pip-6.0.6-py2.7.egg (python 2.7)

[Graham Dumpleton]

On 02/02/2015, at 9:14 PM, Paul Royik <distan...@gmail.com> wrote:

> pip 6.0.6 from /home/username/lib/python2.7/pip-6.0.6-py2.7.egg (python 2.7)

Try:

pip2.7 -v -v -v --log $HOME/pip.log install --user mod_wsgi-httpd

And see if $HOME/pip.log is created.

Graham

[Paul Royik]

There was an error checking the latest version of pip

Traceback (most recent call last):

  File "/home/simamura/lib/python2.7/pip-6.0.6-py2.7.egg/pip/utils/outdated.py", line 129, in pip_version_check

    state.save(pypi_version, current_time)

  File "/home/simamura/lib/python2.7/pip-6.0.6-py2.7.egg/pip/utils/outdated.py", line 76, in save

    with open(self.statefile_path) as statefile:

IOError: [Errno 2] No such file or directory: '/home/simamura/.cache/pip/selfcheck.json'

Collecting mod-wsgi-httpd

  Getting page https://pypi.python.org/simple/mod-wsgi-httpd/

  URLs to search for versions for mod-wsgi-httpd:

  * https://pypi.python.org/simple/mod-wsgi-httpd/

  Getting page https://pypi.python.org/simple/mod-wsgi-httpd/

  Analyzing links from page https://pypi.python.org/simple/mod-wsgi-httpd/

    Found link https://pypi.python.org/packages/source/m/mod_wsgi-httpd/mod_wsgi-httpd-2.4.12.1.tar.gz#md5=15c72c8b53763e764f72b1ee0dffdc6d (from https://pypi.python.org/simple/mod-wsgi-httpd/), version: 2.4.12.1

    Found link https://pypi.python.org/packages/source/m/mod_wsgi-httpd/mod_wsgi-httpd-2.4.12.3.tar.gz#md5=da1c5334623f26ef12acc6023df2053c (from https://pypi.python.org/simple/mod-wsgi-httpd/), version: 2.4.12.3

  Using version 2.4.12.3 (newest of versions: 2.4.12.3, 2.4.12.1)

  Using cached mod_wsgi-httpd-2.4.12.3.tar.gz

  Downloading from URL https://pypi.python.org/packages/source/m/mod_wsgi-httpd/mod_wsgi-httpd-2.4.12.3.tar.gz#md5=da1c5334623f26ef12acc6023df2053c (from https://pypi.python.org/simple/mod-wsgi-httpd/)

  Running setup.py (path:/tmp/pip-build-Ok0zmi/mod-wsgi-httpd/setup.py) egg_info for package mod-wsgi-httpd

    Running command python setup.py egg_info

    apr-1.5.1/

    apr-1.5.1/libapr.mak

    apr-1.5.1/apr.mak

    apr-1.5.1/apr.spec

    apr-1.5.1/docs/

    apr-1.5.1/docs/APRDesign.html

    apr-1.5.1/docs/win32_builds.html

    apr-1.5.1/docs/canonical_filenames.html

    apr-1.5.1/docs/incomplete_types

    apr-1.5.1/docs/non_apr_programs

    apr-1.5.1/docs/pool-design.html

    apr-1.5.1/docs/doxygen.conf

    apr-1.5.1/NWGNUmakefile

    apr-1.5.1/emacs-mode

    apr-1.5.1/test/

    apr-1.5.1/test/testhash.c

    apr-1.5.1/test/testdll.dsp

    apr-1.5.1/test/globalmutexchild.c

    apr-1.5.1/test/internal/

    apr-1.5.1/test/internal/testregex.c

    apr-1.5.1/test/internal/Makefile.in

    apr-1.5.1/test/internal/Makefile.win

    apr-1.5.1/test/internal/testucs.c

    apr-1.5.1/test/testsockopt.c

    apr-1.5.1/test/NWGNUmakefile

    apr-1.5.1/test/testdso.c

    apr-1.5.1/test/teststrnatcmp.c

    apr-1.5.1/test/testshm.c

    apr-1.5.1/test/testutil.c

    apr-1.5.1/test/testglobalmutex.h

    apr-1.5.1/test/testpath.c

    apr-1.5.1/test/testutil.h

    apr-1.5.1/test/NWGNUechod

    apr-1.5.1/test/testtime.c

    apr-1.5.1/test/abts.c

    apr-1.5.1/test/NWGNUaprtest

    apr-1.5.1/test/testargs.c

    apr-1.5.1/test/Makefile.in

    apr-1.5.1/test/echod.c

    apr-1.5.1/test/testdup.c

    apr-1.5.1/test/occhild.c

    apr-1.5.1/test/testuser.c

    apr-1.5.1/test/testsleep.c

    apr-1.5.1/test/testpoll.c

    apr-1.5.1/test/testshm.h

    apr-1.5.1/test/NWGNUsockchild

    apr-1.5.1/test/testcond.c

    apr-1.5.1/test/testvsn.c

    apr-1.5.1/test/testsock.h

    apr-1.5.1/test/testfileinfo.c

    apr-1.5.1/test/testnames.c

    apr-1.5.1/test/testdir.c

    apr-1.5.1/test/sockchild.c

    apr-1.5.1/test/testthread.c

    apr-1.5.1/test/mod_test.c

    apr-1.5.1/test/testsock.c

    apr-1.5.1/test/testtable.c

    apr-1.5.1/test/testlock.c

    apr-1.5.1/test/testprocmutex.c

    apr-1.5.1/test/testpools.c

    apr-1.5.1/test/proc_child.c

    apr-1.5.1/test/testlib.dsp

    apr-1.5.1/test/testproc.c

    apr-1.5.1/test/testlockperf.c

    apr-1.5.1/test/testfmt.c

    apr-1.5.1/test/nw_misc.c

    apr-1.5.1/test/testoc.c

    apr-1.5.1/test/NWGNUtestatmc

    apr-1.5.1/test/testpipe.c

    apr-1.5.1/test/testrand.c

    apr-1.5.1/test/Makefile.win

    apr-1.5.1/test/NWGNUsockperf

    apr-1.5.1/test/sendfile.c

    apr-1.5.1/test/testflock.h

    apr-1.5.1/test/testmutexscope.c

    apr-1.5.1/test/testatomic.c

    apr-1.5.1/test/NWGNUtryread

    apr-1.5.1/test/testall.dsw

    apr-1.5.1/test/NWGNUreadchild

    apr-1.5.1/test/testud.c

    apr-1.5.1/test/abts.h

    apr-1.5.1/test/testlfs.c

    apr-1.5.1/test/testfile.c

    apr-1.5.1/test/tryread.c

    apr-1.5.1/test/NWGNUmod_test

    apr-1.5.1/test/testshmproducer.c

    apr-1.5.1/test/testglobalmutex.c

    apr-1.5.1/test/NWGNUproc_child

    apr-1.5.1/test/testsockets.c

    apr-1.5.1/test/testshmconsumer.c

    apr-1.5.1/test/README

    apr-1.5.1/test/data/

    apr-1.5.1/test/data/mmap_datafile.txt

    apr-1.5.1/test/data/file_datafile.txt

    apr-1.5.1/test/abts_tests.h

    apr-1.5.1/test/readchild.c

    apr-1.5.1/test/NWGNUglobalmutexchild

    apr-1.5.1/test/testflock.c

    apr-1.5.1/test/testfnmatch.c

    apr-1.5.1/test/testmmap.c

    apr-1.5.1/test/sockperf.c

    apr-1.5.1/test/testenv.c

    apr-1.5.1/test/testfilecopy.c

    apr-1.5.1/test/testapp.c

    apr-1.5.1/test/testescape.c

    apr-1.5.1/test/testipsub.c

    apr-1.5.1/test/testtemp.c

    apr-1.5.1/test/teststr.c

    apr-1.5.1/apr.dep

    apr-1.5.1/poll/

    apr-1.5.1/poll/os2/

    apr-1.5.1/poll/os2/pollset.c

    apr-1.5.1/poll/os2/poll.c

    apr-1.5.1/poll/unix/

    apr-1.5.1/poll/unix/z_asio.c

    apr-1.5.1/poll/unix/select.c

    apr-1.5.1/poll/unix/pollset.c

    apr-1.5.1/poll/unix/epoll.c

    apr-1.5.1/poll/unix/pollcb.c

    apr-1.5.1/poll/unix/port.c

    apr-1.5.1/poll/unix/poll.c

    apr-1.5.1/poll/unix/kqueue.c

    apr-1.5.1/passwd/

    apr-1.5.1/passwd/apr_getpass.c

    apr-1.5.1/configure.in

    apr-1.5.1/Makefile.in

    apr-1.5.1/build.conf

    apr-1.5.1/libapr.dsp

    apr-1.5.1/README.cmake

    apr-1.5.1/shmem/

    apr-1.5.1/shmem/os2/

    apr-1.5.1/shmem/os2/shm.c

    apr-1.5.1/shmem/unix/

    apr-1.5.1/shmem/unix/shm.c

    apr-1.5.1/shmem/beos/

    apr-1.5.1/shmem/beos/shm.c

    apr-1.5.1/shmem/win32/

    apr-1.5.1/shmem/win32/shm.c

    apr-1.5.1/time/

    apr-1.5.1/time/unix/

    apr-1.5.1/time/unix/timestr.c

    apr-1.5.1/time/unix/time.c

    apr-1.5.1/time/win32/

    apr-1.5.1/time/win32/timestr.c

    apr-1.5.1/time/win32/time.c

    apr-1.5.1/libapr.dep

    apr-1.5.1/configure

    apr-1.5.1/apr-config.in

    apr-1.5.1/user/

    apr-1.5.1/user/netware/

    apr-1.5.1/user/netware/groupinfo.c

    apr-1.5.1/user/netware/userinfo.c

    apr-1.5.1/user/unix/

    apr-1.5.1/user/unix/groupinfo.c

    apr-1.5.1/user/unix/userinfo.c

    apr-1.5.1/user/win32/

    apr-1.5.1/user/win32/groupinfo.c

    apr-1.5.1/user/win32/userinfo.c

    apr-1.5.1/CHANGES

    apr-1.5.1/CMakeLists.txt

    apr-1.5.1/tables/

    apr-1.5.1/tables/apr_hash.c

    apr-1.5.1/tables/apr_tables.c

    apr-1.5.1/tables/apr_skiplist.c

    apr-1.5.1/config.layout

    apr-1.5.1/support/

    apr-1.5.1/support/unix/

    apr-1.5.1/support/unix/waitio.c

    apr-1.5.1/libapr.rc

    apr-1.5.1/strings/

    apr-1.5.1/strings/apr_fnmatch.c

    apr-1.5.1/strings/apr_cpystrn.c

    apr-1.5.1/strings/apr_strings.c

    apr-1.5.1/strings/apr_snprintf.c

    apr-1.5.1/strings/apr_strnatcmp.c

    apr-1.5.1/strings/apr_strtok.c

    apr-1.5.1/locks/

    apr-1.5.1/locks/os2/

    apr-1.5.1/locks/os2/proc_mutex.c

    apr-1.5.1/locks/os2/thread_mutex.c

    apr-1.5.1/locks/os2/thread_cond.c

    apr-1.5.1/locks/os2/thread_rwlock.c

    apr-1.5.1/locks/netware/

    apr-1.5.1/locks/netware/proc_mutex.c

    apr-1.5.1/locks/netware/thread_mutex.c

    apr-1.5.1/locks/netware/thread_cond.c

    apr-1.5.1/locks/netware/thread_rwlock.c

    apr-1.5.1/locks/unix/

    apr-1.5.1/locks/unix/proc_mutex.c

    apr-1.5.1/locks/unix/thread_mutex.c

    apr-1.5.1/locks/unix/thread_cond.c

    apr-1.5.1/locks/unix/thread_rwlock.c

    apr-1.5.1/locks/unix/global_mutex.c

    apr-1.5.1/locks/beos/

    apr-1.5.1/locks/beos/proc_mutex.c

    apr-1.5.1/locks/beos/thread_mutex.c

    apr-1.5.1/locks/beos/thread_cond.c

    apr-1.5.1/locks/beos/thread_rwlock.c

    apr-1.5.1/locks/win32/

    apr-1.5.1/locks/win32/proc_mutex.c

    apr-1.5.1/locks/win32/thread_mutex.c

    apr-1.5.1/locks/win32/thread_cond.c

    apr-1.5.1/locks/win32/thread_rwlock.c

    apr-1.5.1/Makefile.win

    apr-1.5.1/include/

    apr-1.5.1/include/apr.hnw

    apr-1.5.1/include/apr_network_io.h

    apr-1.5.1/include/apr_mmap.h

    apr-1.5.1/include/apr_file_info.h

    apr-1.5.1/include/apr_proc_mutex.h

    apr-1.5.1/include/apr_errno.h

    apr-1.5.1/include/apr_global_mutex.h

    apr-1.5.1/include/apr_env.h

    apr-1.5.1/include/apr_escape.h

    apr-1.5.1/include/apr.hwc

    apr-1.5.1/include/arch/

    apr-1.5.1/include/arch/os2/

    apr-1.5.1/include/arch/os2/apr_arch_thread_cond.h

    apr-1.5.1/include/arch/os2/apr_arch_thread_mutex.h

    apr-1.5.1/include/arch/os2/apr_arch_proc_mutex.h

    apr-1.5.1/include/arch/os2/apr_arch_dso.h

    apr-1.5.1/include/arch/os2/apr_arch_inherit.h

    apr-1.5.1/include/arch/os2/apr_arch_os2calls.h

    apr-1.5.1/include/arch/os2/apr_arch_networkio.h

    apr-1.5.1/include/arch/os2/apr_arch_threadproc.h

    apr-1.5.1/include/arch/os2/apr_arch_thread_rwlock.h

    apr-1.5.1/include/arch/os2/apr_arch_file_io.h

    apr-1.5.1/include/arch/netware/

    apr-1.5.1/include/arch/netware/apr_arch_thread_cond.h

    apr-1.5.1/include/arch/netware/apr_arch_global_mutex.h

    apr-1.5.1/include/arch/netware/apr_arch_thread_mutex.h

    apr-1.5.1/include/arch/netware/apr_arch_internal_time.h

    apr-1.5.1/include/arch/netware/apr_arch_proc_mutex.h

    apr-1.5.1/include/arch/netware/apr_private.h

    apr-1.5.1/include/arch/netware/apr_arch_dso.h

    apr-1.5.1/include/arch/netware/apr_arch_networkio.h

    apr-1.5.1/include/arch/netware/apr_arch_threadproc.h

    apr-1.5.1/include/arch/netware/apr_arch_pre_nw.h

    apr-1.5.1/include/arch/netware/apr_arch_thread_rwlock.h

    apr-1.5.1/include/arch/netware/apr_arch_file_io.h

    apr-1.5.1/include/arch/darwin/

    apr-1.5.1/include/arch/unix/

    apr-1.5.1/include/arch/unix/apr_arch_thread_cond.h

    apr-1.5.1/include/arch/unix/apr_arch_global_mutex.h

    apr-1.5.1/include/arch/unix/apr_arch_thread_mutex.h

    apr-1.5.1/include/arch/unix/apr_arch_internal_time.h

    apr-1.5.1/include/arch/unix/apr_arch_proc_mutex.h

    apr-1.5.1/include/arch/unix/apr_private.h.in

    apr-1.5.1/include/arch/unix/apr_arch_dso.h

    apr-1.5.1/include/arch/unix/apr_arch_misc.h

    apr-1.5.1/include/arch/unix/apr_arch_shm.h

    apr-1.5.1/include/arch/unix/apr_arch_inherit.h

    apr-1.5.1/include/arch/unix/apr_arch_poll_private.h

    apr-1.5.1/include/arch/unix/apr_arch_networkio.h

    apr-1.5.1/include/arch/unix/apr_arch_threadproc.h

    apr-1.5.1/include/arch/unix/apr_arch_thread_rwlock.h

    apr-1.5.1/include/arch/unix/apr_arch_atomic.h

    apr-1.5.1/include/arch/unix/apr_arch_file_io.h

    apr-1.5.1/include/arch/beos/

    apr-1.5.1/include/arch/beos/apr_arch_thread_cond.h

    apr-1.5.1/include/arch/beos/apr_arch_thread_mutex.h

    apr-1.5.1/include/arch/beos/apr_arch_proc_mutex.h

    apr-1.5.1/include/arch/beos/apr_arch_dso.h

    apr-1.5.1/include/arch/beos/apr_arch_threadproc.h

    apr-1.5.1/include/arch/beos/apr_arch_thread_rwlock.h

    apr-1.5.1/include/arch/aix/

    apr-1.5.1/include/arch/aix/apr_arch_dso.h

    apr-1.5.1/include/arch/win32/

    apr-1.5.1/include/arch/win32/apr_arch_thread_cond.h

    apr-1.5.1/include/arch/win32/apr_arch_thread_mutex.h

    apr-1.5.1/include/arch/win32/apr_arch_proc_mutex.h

    apr-1.5.1/include/arch/win32/apr_private.h

    apr-1.5.1/include/arch/win32/apr_arch_dso.h

    apr-1.5.1/include/arch/win32/apr_arch_atime.h

    apr-1.5.1/include/arch/win32/apr_arch_misc.h

    apr-1.5.1/include/arch/win32/apr_arch_inherit.h

    apr-1.5.1/include/arch/win32/apr_arch_networkio.h

    apr-1.5.1/include/arch/win32/apr_arch_threadproc.h

    apr-1.5.1/include/arch/win32/apr_arch_thread_rwlock.h

    apr-1.5.1/include/arch/win32/apr_arch_utf8.h

    apr-1.5.1/include/arch/win32/apr_arch_file_io.h

    apr-1.5.1/include/arch/win32/apr_dbg_win32_handles.h

    apr-1.5.1/include/arch/os390/

    apr-1.5.1/include/arch/os390/apr_arch_dso.h

    apr-1.5.1/include/arch/apr_private_common.h

    apr-1.5.1/include/apr_random.h

    apr-1.5.1/include/apr_thread_mutex.h

    apr-1.5.1/include/apr_support.h

    apr-1.5.1/include/apr.h.in

    apr-1.5.1/include/apr.hw

    apr-1.5.1/include/apr_user.h

    apr-1.5.1/include/apr_lib.h

    apr-1.5.1/include/apr_portable.h

    apr-1.5.1/include/apr_thread_proc.h

    apr-1.5.1/include/apr_fnmatch.h

    apr-1.5.1/include/apr_file_io.h

    apr-1.5.1/include/apr_getopt.h

    apr-1.5.1/include/apr_hash.h

    apr-1.5.1/include/apr_shm.h

    apr-1.5.1/include/apr_skiplist.h

    apr-1.5.1/include/apr_allocator.h

    apr-1.5.1/include/apr_poll.h

    apr-1.5.1/include/apr_time.h

    apr-1.5.1/include/apr_strings.h

    apr-1.5.1/include/apr_ring.h

    apr-1.5.1/include/apr_thread_rwlock.h

    apr-1.5.1/include/apr_version.h

    apr-1.5.1/include/apr_inherit.h

    apr-1.5.1/include/private/

    apr-1.5.1/include/apr_thread_cond.h

    apr-1.5.1/include/apr_pools.h

    apr-1.5.1/include/apr_signal.h

    apr-1.5.1/include/apr_atomic.h

    apr-1.5.1/include/apr_want.h

    apr-1.5.1/include/apr_tables.h

    apr-1.5.1/include/apr_dso.h

    apr-1.5.1/include/apr_general.h

    apr-1.5.1/network_io/

    apr-1.5.1/network_io/os2/

    apr-1.5.1/network_io/os2/sockopt.c

    apr-1.5.1/network_io/os2/sendrecv_udp.c

    apr-1.5.1/network_io/os2/os2calls.c

    apr-1.5.1/network_io/os2/socket_util.c

    apr-1.5.1/network_io/os2/sockaddr.c

    apr-1.5.1/network_io/os2/inet_ntop.c

    apr-1.5.1/network_io/os2/inet_pton.c

    apr-1.5.1/network_io/os2/sendrecv.c

    apr-1.5.1/network_io/os2/sockets.c

    apr-1.5.1/network_io/unix/

    apr-1.5.1/network_io/unix/sockopt.c

    apr-1.5.1/network_io/unix/multicast.c

    apr-1.5.1/network_io/unix/socket_util.c

    apr-1.5.1/network_io/unix/sockaddr.c

    apr-1.5.1/network_io/unix/inet_ntop.c

    apr-1.5.1/network_io/unix/inet_pton.c

    apr-1.5.1/network_io/unix/sendrecv.c

    apr-1.5.1/network_io/unix/sockets.c

    apr-1.5.1/network_io/beos/

    apr-1.5.1/network_io/beos/sendrecv.c

    apr-1.5.1/network_io/beos/socketcommon.c

    apr-1.5.1/network_io/win32/

    apr-1.5.1/network_io/win32/sockopt.c

    apr-1.5.1/network_io/win32/sendrecv.c

    apr-1.5.1/network_io/win32/sockets.c

    apr-1.5.1/apr.pc.in

    apr-1.5.1/atomic/

    apr-1.5.1/atomic/netware/

    apr-1.5.1/atomic/netware/apr_atomic.c

    apr-1.5.1/atomic/unix/

    apr-1.5.1/atomic/unix/builtins.c

    apr-1.5.1/atomic/unix/ppc.c

    apr-1.5.1/atomic/unix/s390.c

    apr-1.5.1/atomic/unix/mutex.c

    apr-1.5.1/atomic/unix/ia32.c

    apr-1.5.1/atomic/unix/solaris.c

    apr-1.5.1/atomic/win32/

    apr-1.5.1/atomic/win32/apr_atomic.c

    apr-1.5.1/atomic/os390/

    apr-1.5.1/atomic/os390/atomic.c

    apr-1.5.1/memory/

    apr-1.5.1/memory/unix/

    apr-1.5.1/memory/unix/apr_pools.c

    apr-1.5.1/misc/

    apr-1.5.1/misc/netware/

    apr-1.5.1/misc/netware/rand.c

    apr-1.5.1/misc/netware/libprews.c

    apr-1.5.1/misc/netware/aprlib.def

    apr-1.5.1/misc/netware/charset.c

    apr-1.5.1/misc/netware/apr.xdc

    apr-1.5.1/misc/netware/start.c

    apr-1.5.1/misc/unix/

    apr-1.5.1/misc/unix/rand.c

    apr-1.5.1/misc/unix/env.c

    apr-1.5.1/misc/unix/getopt.c

    apr-1.5.1/misc/unix/charset.c

    apr-1.5.1/misc/unix/otherchild.c

    apr-1.5.1/misc/unix/errorcodes.c

    apr-1.5.1/misc/unix/randbyte_os2.inc

    apr-1.5.1/misc/unix/start.c

    apr-1.5.1/misc/unix/version.c

    apr-1.5.1/misc/win32/

    apr-1.5.1/misc/win32/misc.c

    apr-1.5.1/misc/win32/rand.c

    apr-1.5.1/misc/win32/internal.c

    apr-1.5.1/misc/win32/env.c

    apr-1.5.1/misc/win32/apr_app.c

    apr-1.5.1/misc/win32/charset.c

    apr-1.5.1/misc/win32/utf8.c

    apr-1.5.1/misc/win32/start.c

    apr-1.5.1/random/

    apr-1.5.1/random/unix/

    apr-1.5.1/random/unix/apr_random.c

    apr-1.5.1/random/unix/sha2.h

    apr-1.5.1/random/unix/sha2.c

    apr-1.5.1/random/unix/sha2_glue.c

    apr-1.5.1/mmap/

    apr-1.5.1/mmap/unix/

    apr-1.5.1/mmap/unix/common.c

    apr-1.5.1/mmap/unix/mmap.c

    apr-1.5.1/mmap/win32/

    apr-1.5.1/mmap/win32/mmap.c

    apr-1.5.1/apr.dsw

    apr-1.5.1/file_io/

    apr-1.5.1/file_io/os2/

    apr-1.5.1/file_io/os2/fileacc.c

    apr-1.5.1/file_io/os2/mktemp.c

    apr-1.5.1/file_io/os2/dir_make_recurse.c

    apr-1.5.1/file_io/os2/filepath_util.c

    apr-1.5.1/file_io/os2/flock.c

    apr-1.5.1/file_io/os2/seek.c

    apr-1.5.1/file_io/os2/filestat.c

    apr-1.5.1/file_io/os2/pipe.c

    apr-1.5.1/file_io/os2/readwrite.c

    apr-1.5.1/file_io/os2/filesys.c

    apr-1.5.1/file_io/os2/maperrorcode.c

    apr-1.5.1/file_io/os2/fullrw.c

    apr-1.5.1/file_io/os2/filepath.c

    apr-1.5.1/file_io/os2/filedup.c

    apr-1.5.1/file_io/os2/open.c

    apr-1.5.1/file_io/os2/tempdir.c

    apr-1.5.1/file_io/os2/buffer.c

    apr-1.5.1/file_io/os2/copy.c

    apr-1.5.1/file_io/os2/dir.c

    apr-1.5.1/file_io/netware/

    apr-1.5.1/file_io/netware/mktemp.c

    apr-1.5.1/file_io/netware/flock.c

    apr-1.5.1/file_io/netware/filestat.c

    apr-1.5.1/file_io/netware/pipe.c

    apr-1.5.1/file_io/netware/filesys.c

    apr-1.5.1/file_io/netware/filepath.c

    apr-1.5.1/file_io/unix/

    apr-1.5.1/file_io/unix/fileacc.c

    apr-1.5.1/file_io/unix/mktemp.c

    apr-1.5.1/file_io/unix/filepath_util.c

    apr-1.5.1/file_io/unix/flock.c

    apr-1.5.1/file_io/unix/seek.c

    apr-1.5.1/file_io/unix/filestat.c

    apr-1.5.1/file_io/unix/pipe.c

    apr-1.5.1/file_io/unix/readwrite.c

    apr-1.5.1/file_io/unix/fullrw.c

    apr-1.5.1/file_io/unix/filepath.c

    apr-1.5.1/file_io/unix/filedup.c

    apr-1.5.1/file_io/unix/open.c

    apr-1.5.1/file_io/unix/tempdir.c

    apr-1.5.1/file_io/unix/buffer.c

    apr-1.5.1/file_io/unix/copy.c

    apr-1.5.1/file_io/unix/dir.c

    apr-1.5.1/file_io/win32/

    apr-1.5.1/file_io/win32/flock.c

    apr-1.5.1/file_io/win32/seek.c

    apr-1.5.1/file_io/win32/filestat.c

    apr-1.5.1/file_io/win32/pipe.c

    apr-1.5.1/file_io/win32/readwrite.c

    apr-1.5.1/file_io/win32/filesys.c

    apr-1.5.1/file_io/win32/filepath.c

    apr-1.5.1/file_io/win32/filedup.c

    apr-1.5.1/file_io/win32/open.c

    apr-1.5.1/file_io/win32/buffer.c

    apr-1.5.1/file_io/win32/dir.c

    apr-1.5.1/threadproc/

    apr-1.5.1/threadproc/os2/

    apr-1.5.1/threadproc/os2/threadpriv.c

    apr-1.5.1/threadproc/os2/thread.c

    apr-1.5.1/threadproc/os2/proc.c

    apr-1.5.1/threadproc/os2/signals.c

    apr-1.5.1/threadproc/netware/

    apr-1.5.1/threadproc/netware/threadpriv.c

    apr-1.5.1/threadproc/netware/procsup.c

    apr-1.5.1/threadproc/netware/thread.c

    apr-1.5.1/threadproc/netware/proc.c

    apr-1.5.1/threadproc/netware/signals.c

    apr-1.5.1/threadproc/unix/

    apr-1.5.1/threadproc/unix/threadpriv.c

    apr-1.5.1/threadproc/unix/procsup.c

    apr-1.5.1/threadproc/unix/thread.c

    apr-1.5.1/threadproc/unix/proc.c

    apr-1.5.1/threadproc/unix/signals.c

    apr-1.5.1/threadproc/beos/

    apr-1.5.1/threadproc/beos/apr_proc_stub.c

    apr-1.5.1/threadproc/beos/threadpriv.c

    apr-1.5.1/threadproc/beos/thread.c

    apr-1.5.1/threadproc/beos/threadproc_common.c

    apr-1.5.1/threadproc/beos/proc.c

    apr-1.5.1/threadproc/win32/

    apr-1.5.1/threadproc/win32/threadpriv.c

    apr-1.5.1/threadproc/win32/thread.c

    apr-1.5.1/threadproc/win32/proc.c

    apr-1.5.1/threadproc/win32/signals.c

    apr-1.5.1/LICENSE

    apr-1.5.1/build/

    apr-1.5.1/build/lineends.pl

    apr-1.5.1/build/libaprapp.mak

    apr-1.5.1/build/NWGNUmakefile

    apr-1.5.1/build/win32ver.awk

    apr-1.5.1/build/run-gcov.sh

    apr-1.5.1/build/apr_common.m4

    apr-1.5.1/build/config.guess

    apr-1.5.1/build/aprapp.mak

    apr-1.5.1/build/MakeEtags

    apr-1.5.1/build/apr_rules.mk.in

    apr-1.5.1/build/NWGNUhead.inc

    apr-1.5.1/build/libtool.m4

    apr-1.5.1/build/make_var_export.awk

    apr-1.5.1/build/buildcheck.sh

    apr-1.5.1/build/ltmain.sh

    apr-1.5.1/build/PrintPath

    apr-1.5.1/build/get-version.sh

    apr-1.5.1/build/apr_win32.m4

    apr-1.5.1/build/pkg/

    apr-1.5.1/build/pkg/buildpkg.sh

    apr-1.5.1/build/pkg/README

    apr-1.5.1/build/pkg/pkginfo.in

    apr-1.5.1/build/lt~obsolete.m4

    apr-1.5.1/build/config.sub

    apr-1.5.1/build/mkdir.sh

    apr-1.5.1/build/cvtdsp.pl

    apr-1.5.1/build/gen-build.py

    apr-1.5.1/build/nw_ver.awk

    apr-1.5.1/build/NWGNUtail.inc

    apr-1.5.1/build/fixwin32mak.pl

    apr-1.5.1/build/make_nw_export.awk

    apr-1.5.1/build/prelibaprapp.dsp

    apr-1.5.1/build/libaprapp.dep

    apr-1.5.1/build/find_apr.m4

    apr-1.5.1/build/apr_network.m4

    apr-1.5.1/build/ltsugar.m4

    apr-1.5.1/build/NWGNUenvironment.inc

    apr-1.5.1/build/make_exports.awk

    apr-1.5.1/build/aplibtool.c

    apr-1.5.1/build/ltoptions.m4

    apr-1.5.1/build/rpm/

    apr-1.5.1/build/rpm/apr.spec.in

    apr-1.5.1/build/ltversion.m4

    apr-1.5.1/build/apr_hints.m4

    apr-1.5.1/build/install.sh

    apr-1.5.1/build/preaprapp.dsp

    apr-1.5.1/build/apr_threads.m4

    apr-1.5.1/build/aprapp.dep

    apr-1.5.1/build/libaprapp.dsp

    apr-1.5.1/build/nw_export.inc

    apr-1.5.1/build/aprapp.dsp

    apr-1.5.1/build/nw_make_header.awk

    apr-1.5.1/build/jlibtool.c

    apr-1.5.1/helpers/

    apr-1.5.1/helpers/apr_rename.pl

    apr-1.5.1/README

    apr-1.5.1/apr.dsp

    apr-1.5.1/NOTICE

    apr-1.5.1/buildconf

    apr-1.5.1/build-outputs.mk

    apr-1.5.1/dso/

    apr-1.5.1/dso/os2/

    apr-1.5.1/dso/os2/dso.c

    apr-1.5.1/dso/netware/

    apr-1.5.1/dso/netware/dso.c

    apr-1.5.1/dso/unix/

    apr-1.5.1/dso/unix/dso.c

    apr-1.5.1/dso/beos/

    apr-1.5.1/dso/beos/dso.c

    apr-1.5.1/dso/aix/

    apr-1.5.1/dso/aix/dso.c

    apr-1.5.1/dso/win32/

    apr-1.5.1/dso/win32/dso.c

    apr-1.5.1/dso/os390/

    apr-1.5.1/dso/os390/dso.c

    apr-1.5.1/tools/

    apr-1.5.1/tools/gen_test_char.c

    apr-1.5.1/encoding/

    apr-1.5.1/encoding/apr_escape.c

    sh: ./configure: Permission denied

    Downloading http://www.us.apache.org/dist/apr/apr-1.5.1.tar.gz

    Downloading http://www.us.apache.org/dist/apr/apr-util-1.5.4.tar.gz

    Downloading http://www.us.apache.org/dist/httpd/httpd-2.4.12.tar.gz

    Downloading http://downloads.sourceforge.net/project/pcre/pcre/8.36/pcre-8.36.tar.gz

    Traceback (most recent call last):

      File "<string>", line 20, in <module>

      File "/tmp/pip-build-Ok0zmi/mod-wsgi-httpd/setup.py", line 77, in <module>

        raise RuntimeError('Failed to build APR.')

    RuntimeError: Failed to build APR.

    Complete output from command python setup.py egg_info:

    apr-1.5.1/

    

    apr-1.5.1/libapr.mak

    

    apr-1.5.1/apr.mak

    

    apr-1.5.1/apr.spec

    

    apr-1.5.1/docs/

    

    apr-1.5.1/docs/APRDesign.html

    

    apr-1.5.1/docs/win32_builds.html

    

    apr-1.5.1/docs/canonical_filenames.html

    

    apr-1.5.1/docs/incomplete_types

    

    apr-1.5.1/docs/non_apr_programs

    

    apr-1.5.1/docs/pool-design.html

    

    apr-1.5.1/docs/doxygen.conf

    

    apr-1.5.1/NWGNUmakefile

    

    apr-1.5.1/emacs-mode

    

    apr-1.5.1/test/

    

    apr-1.5.1/test/testhash.c

    

    apr-1.5.1/test/testdll.dsp

    

    apr-1.5.1/test/globalmutexchild.c

    

    apr-1.5.1/test/internal/

    

    apr-1.5.1/test/internal/testregex.c

    

    apr-1.5.1/test/internal/Makefile.in

    

    apr-1.5.1/test/internal/Makefile.win

    

    apr-1.5.1/test/internal/testucs.c

    

    apr-1.5.1/test/testsockopt.c

    

    apr-1.5.1/test/NWGNUmakefile

    

    apr-1.5.1/test/testdso.c

    

    apr-1.5.1/test/teststrnatcmp.c

    

    apr-1.5.1/test/testshm.c

    

    apr-1.5.1/test/testutil.c

    

    apr-1.5.1/test/testglobalmutex.h

    

    apr-1.5.1/test/testpath.c

    

    apr-1.5.1/test/testutil.h

    

    apr-1.5.1/test/NWGNUechod

    

    apr-1.5.1/test/testtime.c

    

    apr-1.5.1/test/abts.c

    

    apr-1.5.1/test/NWGNUaprtest

    

    apr-1.5.1/test/testargs.c

    

    apr-1.5.1/test/Makefile.in

    

    apr-1.5.1/test/echod.c

    

    apr-1.5.1/test/testdup.c

    

    apr-1.5.1/test/occhild.c

    

    apr-1.5.1/test/testuser.c

    

    apr-1.5.1/test/testsleep.c

    

    apr-1.5.1/test/testpoll.c

    

    apr-1.5.1/test/testshm.h

    

    apr-1.5.1/test/NWGNUsockchild

    

    apr-1.5.1/test/testcond.c

    

    apr-1.5.1/test/testvsn.c

    

    apr-1.5.1/test/testsock.h

    

    apr-1.5.1/test/testfileinfo.c

    

    apr-1.5.1/test/testnames.c

    

    apr-1.5.1/test/testdir.c

    

    apr-1.5.1/test/sockchild.c

    

    apr-1.5.1/test/testthread.c

    

    apr-1.5.1/test/mod_test.c

    

    apr-1.5.1/test/testsock.c

    

    apr-1.5.1/test/testtable.c

    

    apr-1.5.1/test/testlock.c

    

    apr-1.5.1/test/testprocmutex.c

    

    apr-1.5.1/test/testpools.c

    

    apr-1.5.1/test/proc_child.c

    

    apr-1.5.1/test/testlib.dsp

    

    apr-1.5.1/test/testproc.c

    

    apr-1.5.1/test/testlockperf.c

    

    apr-1.5.1/test/testfmt.c

    

    apr-1.5.1/test/nw_misc.c

    

    apr-1.5.1/test/testoc.c

    

    apr-1.5.1/test/NWGNUtestatmc

    

    apr-1.5.1/test/testpipe.c

    

    apr-1.5.1/test/testrand.c

    

    apr-1.5.1/test/Makefile.win

    

    apr-1.5.1/test/NWGNUsockperf

    

    apr-1.5.1/test/sendfile.c

    

    apr-1.5.1/test/testflock.h

    

    apr-1.5.1/test/testmutexscope.c

    

    apr-1.5.1/test/testatomic.c

    

    apr-1.5.1/test/NWGNUtryread

    

    apr-1.5.1/test/testall.dsw

    

    apr-1.5.1/test/NWGNUreadchild

    

    apr-1.5.1/test/testud.c

    

    apr-1.5.1/test/abts.h

    

    apr-1.5.1/test/testlfs.c

    

    apr-1.5.1/test/testfile.c

    

    apr-1.5.1/test/tryread.c

    

    apr-1.5.1/test/NWGNUmod_test

    

    apr-1.5.1/test/testshmproducer.c

    

    apr-1.5.1/test/testglobalmutex.c

    

    apr-1.5.1/test/NWGNUproc_child

    

    apr-1.5.1/test/testsockets.c

    

    apr-1.5.1/test/testshmconsumer.c

    

    apr-1.5.1/test/README

    

    apr-1.5.1/test/data/

    

    apr-1.5.1/test/data/mmap_datafile.txt

    

    apr-1.5.1/test/data/file_datafile.txt

    

    apr-1.5.1/test/abts_tests.h

    

    apr-1.5.1/test/readchild.c

    

    apr-1.5.1/test/NWGNUglobalmutexchild

    

    apr-1.5.1/test/testflock.c

    

    apr-1.5.1/test/testfnmatch.c

    

    apr-1.5.1/test/testmmap.c

    

    apr-1.5.1/test/sockperf.c

    

    apr-1.5.1/test/testenv.c

    

    apr-1.5.1/test/testfilecopy.c

    

    apr-1.5.1/test/testapp.c

    

    apr-1.5.1/test/testescape.c

    

    apr-1.5.1/test/testipsub.c

    

    apr-1.5.1/test/testtemp.c

    

    apr-1.5.1/test/teststr.c

    

    apr-1.5.1/apr.dep

    

    apr-1.5.1/poll/

    

    apr-1.5.1/poll/os2/

    

    apr-1.5.1/poll/os2/pollset.c

    

    apr-1.5.1/poll/os2/poll.c

    

    apr-1.5.1/poll/unix/

    

    apr-1.5.1/poll/unix/z_asio.c

    

    apr-1.5.1/poll/unix/select.c

    

    apr-1.5.1/poll/unix/pollset.c

    

    apr-1.5.1/poll/unix/epoll.c

    

    apr-1.5.1/poll/unix/pollcb.c

    

    apr-1.5.1/poll/unix/port.c

    

    apr-1.5.1/poll/unix/poll.c

    

    apr-1.5.1/poll/unix/kqueue.c

    

    apr-1.5.1/passwd/

    

    apr-1.5.1/passwd/apr_getpass.c

    

    apr-1.5.1/configure.in

    

    apr-1.5.1/Makefile.in

    

    apr-1.5.1/build.conf

    

    apr-1.5.1/libapr.dsp

    

    apr-1.5.1/README.cmake

    

    apr-1.5.1/shmem/

    

    apr-1.5.1/shmem/os2/

    

    apr-1.5.1/shmem/os2/shm.c

    

    apr-1.5.1/shmem/unix/

    

    apr-1.5.1/shmem/unix/shm.c

    

    apr-1.5.1/shmem/beos/

    

    apr-1.5.1/shmem/beos/shm.c

    

    apr-1.5.1/shmem/win32/

    

    apr-1.5.1/shmem/win32/shm.c

    

    apr-1.5.1/time/

    

    apr-1.5.1/time/unix/

    

    apr-1.5.1/time/unix/timestr.c

    

    apr-1.5.1/time/unix/time.c

    

    apr-1.5.1/time/win32/

    

    apr-1.5.1/time/win32/timestr.c

    

    apr-1.5.1/time/win32/time.c

    

    apr-1.5.1/libapr.dep

    

    apr-1.5.1/configure

    

    apr-1.5.1/apr-config.in

    

    apr-1.5.1/user/

    

    apr-1.5.1/user/netware/

    

    apr-1.5.1/user/netware/groupinfo.c

    

    apr-1.5.1/user/netware/userinfo.c

    

    apr-1.5.1/user/unix/

    

    apr-1.5.1/user/unix/groupinfo.c

    

    apr-1.5.1/user/unix/userinfo.c

    

    apr-1.5.1/user/win32/

    

    apr-1.5.1/user/win32/groupinfo.c

    

    apr-1.5.1/user/win32/userinfo.c

    

    apr-1.5.1/CHANGES

    

    apr-1.5.1/CMakeLists.txt

    

    apr-1.5.1/tables/

    

    apr-1.5.1/tables/apr_hash.c

    

    apr-1.5.1/tables/apr_tables.c

    

    apr-1.5.1/tables/apr_skiplist.c

    

    apr-1.5.1/config.layout

    

    apr-1.5.1/support/

    

    apr-1.5.1/support/unix/

    

    apr-1.5.1/support/unix/waitio.c

    

    apr-1.5.1/libapr.rc

    

    apr-1.5.1/strings/

    

    apr-1.5.1/strings/apr_fnmatch.c

    

    apr-1.5.1/strings/apr_cpystrn.c

    

    apr-1.5.1/strings/apr_strings.c

    

    apr-1.5.1/strings/apr_snprintf.c

    

    apr-1.5.1/strings/apr_strnatcmp.c

    

    apr-1.5.1/strings/apr_strtok.c

    

    apr-1.5.1/locks/

    

    apr-1.5.1/locks/os2/

    

    apr-1.5.1/locks/os2/proc_mutex.c

    

    apr-1.5.1/locks/os2/thread_mutex.c

    

    apr-1.5.1/locks/os2/thread_cond.c

    

    apr-1.5.1/locks/os2/thread_rwlock.c

    

    apr-1.5.1/locks/netware/

    

    apr-1.5.1/locks/netware/proc_mutex.c

    

    apr-1.5.1/locks/netware/thread_mutex.c

    

    apr-1.5.1/locks/netware/thread_cond.c

    

    apr-1.5.1/locks/netware/thread_rwlock.c

    

    apr-1.5.1/locks/unix/

    

    apr-1.5.1/locks/unix/proc_mutex.c

    

    apr-1.5.1/locks/unix/thread_mutex.c

    

    apr-1.5.1/locks/unix/thread_cond.c

    

    apr-1.5.1/locks/unix/thread_rwlock.c

    

    apr-1.5.1/locks/unix/global_mutex.c

    

    apr-1.5.1/locks/beos/

    

    apr-1.5.1/locks/beos/proc_mutex.c

    

    apr-1.5.1/locks/beos/thread_mutex.c

    

    apr-1.5.1/locks/beos/thread_cond.c

    

    apr-1.5.1/locks/beos/thread_rwlock.c

    

    apr-1.5.1/locks/win32/

    

    apr-1.5.1/locks/win32/proc_mutex.c

    

    apr-1.5.1/locks/win32/thread_mutex.c

    

    apr-1.5.1/locks/win32/thread_cond.c

    

    apr-1.5.1/locks/win32/thread_rwlock.c

    

    apr-1.5.1/Makefile.win

    

    apr-1.5.1/include/

    

    apr-1.5.1/include/apr.hnw

    

    apr-1.5.1/include/apr_network_io.h

    

    apr-1.5.1/include/apr_mmap.h

    

    apr-1.5.1/include/apr_file_info.h

    

    apr-1.5.1/include/apr_proc_mutex.h

    

    apr-1.5.1/include/apr_errno.h

    

    apr-1.5.1/include/apr_global_mutex.h

    

    apr-1.5.1/include/apr_env.h

    

    apr-1.5.1/include/apr_escape.h

    

    apr-1.5.1/include/apr.hwc

    

    apr-1.5.1/include/arch/

    

    apr-1.5.1/include/arch/os2/

    

    apr-1.5.1/include/arch/os2/apr_arch_thread_cond.h

    

    apr-1.5.1/include/arch/os2/apr_arch_thread_mutex.h

    

    apr-1.5.1/include/arch/os2/apr_arch_proc_mutex.h

    

    apr-1.5.1/include/arch/os2/apr_arch_dso.h

    

    apr-1.5.1/include/arch/os2/apr_arch_inherit.h

    

    apr-1.5.1/include/arch/os2/apr_arch_os2calls.h

    

    apr-1.5.1/include/arch/os2/apr_arch_networkio.h

    

    apr-1.5.1/include/arch/os2/apr_arch_threadproc.h

    

    apr-1.5.1/include/arch/os2/apr_arch_thread_rwlock.h

    

    apr-1.5.1/include/arch/os2/apr_arch_file_io.h

    

    apr-1.5.1/include/arch/netware/

    

    apr-1.5.1/include/arch/netware/apr_arch_thread_cond.h

    

    apr-1.5.1/include/arch/netware/apr_arch_global_mutex.h

    

    apr-1.5.1/include/arch/netware/apr_arch_thread_mutex.h

    

    apr-1.5.1/include/arch/netware/apr_arch_internal_time.h

    

    apr-1.5.1/include/arch/netware/apr_arch_proc_mutex.h

    

    apr-1.5.1/include/arch/netware/apr_private.h

    

    apr-1.5.1/include/arch/netware/apr_arch_dso.h

    

    apr-1.5.1/include/arch/netware/apr_arch_networkio.h

    

    apr-1.5.1/include/arch/netware/apr_arch_threadproc.h

    

    apr-1.5.1/include/arch/netware/apr_arch_pre_nw.h

    

    apr-1.5.1/include/arch/netware/apr_arch_thread_rwlock.h

    

    apr-1.5.1/include/arch/netware/apr_arch_file_io.h

    

    apr-1.5.1/include/arch/darwin/

    

    apr-1.5.1/include/arch/unix/

    

    apr-1.5.1/include/arch/unix/apr_arch_thread_cond.h

    

    apr-1.5.1/include/arch/unix/apr_arch_global_mutex.h

    

    apr-1.5.1/include/arch/unix/apr_arch_thread_mutex.h

    

    apr-1.5.1/include/arch/unix/apr_arch_internal_time.h

    

    apr-1.5.1/include/arch/unix/apr_arch_proc_mutex.h

    

    apr-1.5.1/include/arch/unix/apr_private.h.in

    

    apr-1.5.1/include/arch/unix/apr_arch_dso.h

    

    apr-1.5.1/include/arch/unix/apr_arch_misc.h

    

    apr-1.5.1/include/arch/unix/apr_arch_shm.h

    

    apr-1.5.1/include/arch/unix/apr_arch_inherit.h

    

    apr-1.5.1/include/arch/unix/apr_arch_poll_private.h

    

    apr-1.5.1/include/arch/unix/apr_arch_networkio.h

    

    apr-1.5.1/include/arch/unix/apr_arch_threadproc.h

    

    apr-1.5.1/include/arch/unix/apr_arch_thread_rwlock.h

    

    apr-1.5.1/include/arch/unix/apr_arch_atomic.h

    

    apr-1.5.1/include/arch/unix/apr_arch_file_io.h

    

    apr-1.5.1/include/arch/beos/

    

    apr-1.5.1/include/arch/beos/apr_arch_thread_cond.h

    

    apr-1.5.1/include/arch/beos/apr_arch_thread_mutex.h

    

    apr-1.5.1/include/arch/beos/apr_arch_proc_mutex.h

    

    apr-1.5.1/include/arch/beos/apr_arch_dso.h

    

    apr-1.5.1/include/arch/beos/apr_arch_threadproc.h

    

    apr-1.5.1/include/arch/beos/apr_arch_thread_rwlock.h

    

    apr-1.5.1/include/arch/aix/

    

    apr-1.5.1/include/arch/aix/apr_arch_dso.h

    

    apr-1.5.1/include/arch/win32/

    

    apr-1.5.1/include/arch/win32/apr_arch_thread_cond.h

    

    apr-1.5.1/include/arch/win32/apr_arch_thread_mutex.h

    

    apr-1.5.1/include/arch/win32/apr_arch_proc_mutex.h

    

    apr-1.5.1/include/arch/win32/apr_private.h

    

    apr-1.5.1/include/arch/win32/apr_arch_dso.h

    

    apr-1.5.1/include/arch/win32/apr_arch_atime.h

    

    apr-1.5.1/include/arch/win32/apr_arch_misc.h

    

    apr-1.5.1/include/arch/win32/apr_arch_inherit.h

    

    apr-1.5.1/include/arch/win32/apr_arch_networkio.h

    

    apr-1.5.1/include/arch/win32/apr_arch_threadproc.h

    

    apr-1.5.1/include/arch/win32/apr_arch_thread_rwlock.h

    

    apr-1.5.1/include/arch/win32/apr_arch_utf8.h

    

    apr-1.5.1/include/arch/win32/apr_arch_file_io.h

    

    apr-1.5.1/include/arch/win32/apr_dbg_win32_handles.h

    

    apr-1.5.1/include/arch/os390/

    

    apr-1.5.1/include/arch/os390/apr_arch_dso.h

    

    apr-1.5.1/include/arch/apr_private_common.h

    

    apr-1.5.1/include/apr_random.h

    

    apr-1.5.1/include/apr_thread_mutex.h

    

    apr-1.5.1/include/apr_support.h

    

    apr-1.5.1/include/apr.h.in

    

    apr-1.5.1/include/apr.hw

    

    apr-1.5.1/include/apr_user.h

    

    apr-1.5.1/include/apr_lib.h

    

    apr-1.5.1/include/apr_portable.h

    

    apr-1.5.1/include/apr_thread_proc.h

    

    apr-1.5.1/include/apr_fnmatch.h

    

    apr-1.5.1/include/apr_file_io.h

    

    apr-1.5.1/include/apr_getopt.h

    

    apr-1.5.1/include/apr_hash.h

    

    apr-1.5.1/include/apr_shm.h

    

    apr-1.5.1/include/apr_skiplist.h

    

    apr-1.5.1/include/apr_allocator.h

    

    apr-1.5.1/include/apr_poll.h

    

    apr-1.5.1/include/apr_time.h

    

    apr-1.5.1/include/apr_strings.h

    

    apr-1.5.1/include/apr_ring.h

    

    apr-1.5.1/include/apr_thread_rwlock.h

    

    apr-1.5.1/include/apr_version.h

    

    apr-1.5.1/include/apr_inherit.h

    

    apr-1.5.1/include/private/

    

    apr-1.5.1/include/apr_thread_cond.h

    

    apr-1.5.1/include/apr_pools.h

    

    apr-1.5.1/include/apr_signal.h

    

    apr-1.5.1/include/apr_atomic.h

    

    apr-1.5.1/include/apr_want.h

    

    apr-1.5.1/include/apr_tables.h

    

    apr-1.5.1/include/apr_dso.h

    

    apr-1.5.1/include/apr_general.h

    

    apr-1.5.1/network_io/

    

    apr-1.5.1/network_io/os2/

    

    apr-1.5.1/network_io/os2/sockopt.c

    

    apr-1.5.1/network_io/os2/sendrecv_udp.c

    

    apr-1.5.1/network_io/os2/os2calls.c

    

    apr-1.5.1/network_io/os2/socket_util.c

    

    apr-1.5.1/network_io/os2/sockaddr.c

    

    apr-1.5.1/network_io/os2/inet_ntop.c

    

    apr-1.5.1/network_io/os2/inet_pton.c

    

    apr-1.5.1/network_io/os2/sendrecv.c

    

    apr-1.5.1/network_io/os2/sockets.c

    

    apr-1.5.1/network_io/unix/

    

    apr-1.5.1/network_io/unix/sockopt.c

    

    apr-1.5.1/network_io/unix/multicast.c

    

    apr-1.5.1/network_io/unix/socket_util.c

    

    apr-1.5.1/network_io/unix/sockaddr.c

    

    apr-1.5.1/network_io/unix/inet_ntop.c

    

    apr-1.5.1/network_io/unix/inet_pton.c

    

    apr-1.5.1/network_io/unix/sendrecv.c

    

    apr-1.5.1/network_io/unix/sockets.c

    

    apr-1.5.1/network_io/beos/

    

    apr-1.5.1/network_io/beos/sendrecv.c

    

    apr-1.5.1/network_io/beos/socketcommon.c

    

    apr-1.5.1/network_io/win32/

    

    apr-1.5.1/network_io/win32/sockopt.c

    

    apr-1.5.1/network_io/win32/sendrecv.c

    

    apr-1.5.1/network_io/win32/sockets.c

    

    apr-1.5.1/apr.pc.in

    

    apr-1.5.1/atomic/

    

    apr-1.5.1/atomic/netware/

    

    apr-1.5.1/atomic/netware/apr_atomic.c

    

    apr-1.5.1/atomic/unix/

    

    apr-1.5.1/atomic/unix/builtins.c

    

    apr-1.5.1/atomic/unix/ppc.c

    

    apr-1.5.1/atomic/unix/s390.c

    

    apr-1.5.1/atomic/unix/mutex.c

    

    apr-1.5.1/atomic/unix/ia32.c

    

    apr-1.5.1/atomic/unix/solaris.c

    

    apr-1.5.1/atomic/win32/

    

    apr-1.5.1/atomic/win32/apr_atomic.c

    

    apr-1.5.1/atomic/os390/

    

    apr-1.5.1/atomic/os390/atomic.c

    

    apr-1.5.1/memory/

    

    apr-1.5.1/memory/unix/

    

    apr-1.5.1/memory/unix/apr_pools.c

    

    apr-1.5.1/misc/

    

    apr-1.5.1/misc/netware/

    

    apr-1.5.1/misc/netware/rand.c

    

    apr-1.5.1/misc/netware/libprews.c

    

    apr-1.5.1/misc/netware/aprlib.def

    

    apr-1.5.1/misc/netware/charset.c

    

    apr-1.5.1/misc/netware/apr.xdc

    

    apr-1.5.1/misc/netware/start.c

    

    apr-1.5.1/misc/unix/

    

    apr-1.5.1/misc/unix/rand.c

    

    apr-1.5.1/misc/unix/env.c

    

    apr-1.5.1/misc/unix/getopt.c

    

    apr-1.5.1/misc/unix/charset.c

    

    apr-1.5.1/misc/unix/otherchild.c

    

    apr-1.5.1/misc/unix/errorcodes.c

    

    apr-1.5.1/misc/unix/randbyte_os2.inc

    

    apr-1.5.1/misc/unix/start.c

    

    apr-1.5.1/misc/unix/version.c

    

    apr-1.5.1/misc/win32/

    

    apr-1.5.1/misc/win32/misc.c

    

    apr-1.5.1/misc/win32/rand.c

    

    apr-1.5.1/misc/win32/internal.c

    

    apr-1.5.1/misc/win32/env.c

    

    apr-1.5.1/misc/win32/apr_app.c

    

    apr-1.5.1/misc/win32/charset.c

    

    apr-1.5.1/misc/win32/utf8.c

    

    apr-1.5.1/misc/win32/start.c

    

    apr-1.5.1/random/

    

    apr-1.5.1/random/unix/

    

    apr-1.5.1/random/unix/apr_random.c

    

    apr-1.5.1/random/unix/sha2.h

    

    apr-1.5.1/random/unix/sha2.c

    

    apr-1.5.1/random/unix/sha2_glue.c

    

    apr-1.5.1/mmap/

    

    apr-1.5.1/mmap/unix/

    

    apr-1.5.1/mmap/unix/common.c

    

    apr-1.5.1/mmap/unix/mmap.c

    

    apr-1.5.1/mmap/win32/

    

    apr-1.5.1/mmap/win32/mmap.c

    

    apr-1.5.1/apr.dsw

    

    apr-1.5.1/file_io/

    

    apr-1.5.1/file_io/os2/

    

    apr-1.5.1/file_io/os2/fileacc.c

    

    apr-1.5.1/file_io/os2/mktemp.c

    

    apr-1.5.1/file_io/os2/dir_make_recurse.c

    

    apr-1.5.1/file_io/os2/filepath_util.c

    

    apr-1.5.1/file_io/os2/flock.c

    

    apr-1.5.1/file_io/os2/seek.c

    

    apr-1.5.1/file_io/os2/filestat.c

    

    apr-1.5.1/file_io/os2/pipe.c

    

    apr-1.5.1/file_io/os2/readwrite.c

    

    apr-1.5.1/file_io/os2/filesys.c

    

    apr-1.5.1/file_io/os2/maperrorcode.c

    

    apr-1.5.1/file_io/os2/fullrw.c

    

    apr-1.5.1/file_io/os2/filepath.c

    

    apr-1.5.1/file_io/os2/filedup.c

    

    apr-1.5.1/file_io/os2/open.c

    

    apr-1.5.1/file_io/os2/tempdir.c

    

    apr-1.5.1/file_io/os2/buffer.c

    

    apr-1.5.1/file_io/os2/copy.c

    

    apr-1.5.1/file_io/os2/dir.c

    

    apr-1.5.1/file_io/netware/

    

    apr-1.5.1/file_io/netware/mktemp.c

    

    apr-1.5.1/file_io/netware/flock.c

    

    apr-1.5.1/file_io/netware/filestat.c

    

    apr-1.5.1/file_io/netware/pipe.c

    

    apr-1.5.1/file_io/netware/filesys.c

    

    apr-1.5.1/file_io/netware/filepath.c

    

    apr-1.5.1/file_io/unix/

    

    apr-1.5.1/file_io/unix/fileacc.c

    

    apr-1.5.1/file_io/unix/mktemp.c

    

    apr-1.5.1/file_io/unix/filepath_util.c

    

    apr-1.5.1/file_io/unix/flock.c

    

    apr-1.5.1/file_io/unix/seek.c

    

    apr-1.5.1/file_io/unix/filestat.c

    

    apr-1.5.1/file_io/unix/pipe.c

    

    apr-1.5.1/file_io/unix/readwrite.c

    

    apr-1.5.1/file_io/unix/fullrw.c

    

    apr-1.5.1/file_io/unix/filepath.c

    

    apr-1.5.1/file_io/unix/filedup.c

    

    apr-1.5.1/file_io/unix/open.c

    

    apr-1.5.1/file_io/unix/tempdir.c

    

    apr-1.5.1/file_io/unix/buffer.c

    

    apr-1.5.1/file_io/unix/copy.c

    

    apr-1.5.1/file_io/unix/dir.c

    

    apr-1.5.1/file_io/win32/

    

    apr-1.5.1/file_io/win32/flock.c

    

    apr-1.5.1/file_io/win32/seek.c

    

    apr-1.5.1/file_io/win32/filestat.c

    

    apr-1.5.1/file_io/win32/pipe.c

    

    apr-1.5.1/file_io/win32/readwrite.c

    

    apr-1.5.1/file_io/win32/filesys.c

    

    apr-1.5.1/file_io/win32/filepath.c

    

    apr-1.5.1/file_io/win32/filedup.c

    

    apr-1.5.1/file_io/win32/open.c

    

    apr-1.5.1/file_io/win32/buffer.c

    

    apr-1.5.1/file_io/win32/dir.c

    

    apr-1.5.1/threadproc/

    

    apr-1.5.1/threadproc/os2/

    

    apr-1.5.1/threadproc/os2/threadpriv.c

    

    apr-1.5.1/threadproc/os2/thread.c

    

    apr-1.5.1/threadproc/os2/proc.c

    

    apr-1.5.1/threadproc/os2/signals.c

    

    apr-1.5.1/threadproc/netware/

    

    apr-1.5.1/threadproc/netware/threadpriv.c

    

    apr-1.5.1/threadproc/netware/procsup.c

    

    apr-1.5.1/threadproc/netware/thread.c

    

    apr-1.5.1/threadproc/netware/proc.c

    

    apr-1.5.1/threadproc/netware/signals.c

    

    apr-1.5.1/threadproc/unix/

    

    apr-1.5.1/threadproc/unix/threadpriv.c

    

    apr-1.5.1/threadproc/unix/procsup.c

    

    apr-1.5.1/threadproc/unix/thread.c

    

    apr-1.5.1/threadproc/unix/proc.c

    

    apr-1.5.1/threadproc/unix/signals.c

    

    apr-1.5.1/threadproc/beos/

    

    apr-1.5.1/threadproc/beos/apr_proc_stub.c

    

    apr-1.5.1/threadproc/beos/threadpriv.c

    

    apr-1.5.1/threadproc/beos/thread.c

    

    apr-1.5.1/threadproc/beos/threadproc_common.c

    

    apr-1.5.1/threadproc/beos/proc.c

    

    apr-1.5.1/threadproc/win32/

    

    apr-1.5.1/threadproc/win32/threadpriv.c

    

    apr-1.5.1/threadproc/win32/thread.c

    

    apr-1.5.1/threadproc/win32/proc.c

    

    apr-1.5.1/threadproc/win32/signals.c

    

    apr-1.5.1/LICENSE

    

    apr-1.5.1/build/

    

    apr-1.5.1/build/lineends.pl

    

    apr-1.5.1/build/libaprapp.mak

    

    apr-1.5.1/build/NWGNUmakefile

    

    apr-1.5.1/build/win32ver.awk

    

    apr-1.5.1/build/run-gcov.sh

    

    apr-1.5.1/build/apr_common.m4

    

    apr-1.5.1/build/config.guess

    

    apr-1.5.1/build/aprapp.mak

    

    apr-1.5.1/build/MakeEtags

    

    apr-1.5.1/build/apr_rules.mk.in

    

    apr-1.5.1/build/NWGNUhead.inc

    

    apr-1.5.1/build/libtool.m4

    

    apr-1.5.1/build/make_var_export.awk

    

    apr-1.5.1/build/buildcheck.sh

    

    apr-1.5.1/build/ltmain.sh

    

    apr-1.5.1/build/PrintPath

    

    apr-1.5.1/build/get-version.sh

    

    apr-1.5.1/build/apr_win32.m4

    

    apr-1.5.1/build/pkg/

    

    apr-1.5.1/build/pkg/buildpkg.sh

    

    apr-1.5.1/build/pkg/README

    

    apr-1.5.1/build/pkg/pkginfo.in

    

    apr-1.5.1/build/lt~obsolete.m4

    

    apr-1.5.1/build/config.sub

    

    apr-1.5.1/build/mkdir.sh

    

    apr-1.5.1/build/cvtdsp.pl

    

    apr-1.5.1/build/gen-build.py

    

    apr-1.5.1/build/nw_ver.awk

    

    apr-1.5.1/build/NWGNUtail.inc

    

    apr-1.5.1/build/fixwin32mak.pl

    

    apr-1.5.1/build/make_nw_export.awk

    

    apr-1.5.1/build/prelibaprapp.dsp

    

    apr-1.5.1/build/libaprapp.dep

    

    apr-1.5.1/build/find_apr.m4

    

    apr-1.5.1/build/apr_network.m4

    

    apr-1.5.1/build/ltsugar.m4

    

    apr-1.5.1/build/NWGNUenvironment.inc

    

    apr-1.5.1/build/make_exports.awk

    

    apr-1.5.1/build/aplibtool.c

    

    apr-1.5.1/build/ltoptions.m4

    

    apr-1.5.1/build/rpm/

    

    apr-1.5.1/build/rpm/apr.spec.in

    

    apr-1.5.1/build/ltversion.m4

    

    apr-1.5.1/build/apr_hints.m4

    

    apr-1.5.1/build/install.sh

    

    apr-1.5.1/build/preaprapp.dsp

    

    apr-1.5.1/build/apr_threads.m4

    

    apr-1.5.1/build/aprapp.dep

    

    apr-1.5.1/build/libaprapp.dsp

    

    apr-1.5.1/build/nw_export.inc

    

    apr-1.5.1/build/aprapp.dsp

    

    apr-1.5.1/build/nw_make_header.awk

    

    apr-1.5.1/build/jlibtool.c

    

    apr-1.5.1/helpers/

    

    apr-1.5.1/helpers/apr_rename.pl

    

    apr-1.5.1/README

    

    apr-1.5.1/apr.dsp

    

    apr-1.5.1/NOTICE

    

    apr-1.5.1/buildconf

    

    apr-1.5.1/build-outputs.mk

    

    apr-1.5.1/dso/

    

    apr-1.5.1/dso/os2/

    

    apr-1.5.1/dso/os2/dso.c

    

    apr-1.5.1/dso/netware/

    

    apr-1.5.1/dso/netware/dso.c

    

    apr-1.5.1/dso/unix/

    

    apr-1.5.1/dso/unix/dso.c

    

    apr-1.5.1/dso/beos/

    

    apr-1.5.1/dso/beos/dso.c

    

    apr-1.5.1/dso/aix/

    

    apr-1.5.1/dso/aix/dso.c

    

    apr-1.5.1/dso/win32/

    

    apr-1.5.1/dso/win32/dso.c

    

    apr-1.5.1/dso/os390/

    

    apr-1.5.1/dso/os390/dso.c

    

    apr-1.5.1/tools/

    

    apr-1.5.1/tools/gen_test_char.c

    

    apr-1.5.1/encoding/

    

    apr-1.5.1/encoding/apr_escape.c

    

    sh: ./configure: Permission denied

    

    Downloading http://www.us.apache.org/dist/apr/apr-1.5.1.tar.gz

    

    Downloading http://www.us.apache.org/dist/apr/apr-util-1.5.4.tar.gz

    

    Downloading http://www.us.apache.org/dist/httpd/httpd-2.4.12.tar.gz

    

    Downloading http://downloads.sourceforge.net/project/pcre/pcre/8.36/pcre-8.36.tar.gz

    

    Traceback (most recent call last):

    

      File "<string>", line 20, in <module>

    

      File "/tmp/pip-build-Ok0zmi/mod-wsgi-httpd/setup.py", line 77, in <module>

    

        raise RuntimeError('Failed to build APR.')

    

    RuntimeError: Failed to build APR.

    

    ----------------------------------------

    Cleaning up...

    Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-Ok0zmi/mod-wsgi-httpd

    Exception information:

    Traceback (most recent call last):

      File "/home/simamura/lib/python2.7/pip-6.0.6-py2.7.egg/pip/basecommand.py", line 232, in main

        status = self.run(options, args)

      File "/home/simamura/lib/python2.7/pip-6.0.6-py2.7.egg/pip/commands/install.py", line 339, in run

        requirement_set.prepare_files(finder)

      File "/home/simamura/lib/python2.7/pip-6.0.6-py2.7.egg/pip/req/req_set.py", line 385, in prepare_files

        req_to_install.run_egg_info()

      File "/home/simamura/lib/python2.7/pip-6.0.6-py2.7.egg/pip/req/req_install.py", line 356, in run_egg_info

        command_desc='python setup.py egg_info')

      File "/home/simamura/lib/python2.7/pip-6.0.6-py2.7.egg/pip/utils/__init__.py", line 746, in call_subprocess

        % (command_desc, proc.returncode, cwd))

    InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-Ok0zmi/mod-wsgi-httpd

[Graham Dumpleton]

Something is still screwing with file permissions when the tar ball is unpacked.

This time I changed to using normal 'tar' command to extract the tar ball to ensure that Python tarfile module couldn't cause any issues.

You said umask command gave:

    0002

which means it should be fine.

I can only imagine that web faction is doing some weird things with access control lists or something which prevents you having executable scripts in /tmp/.

Try instead:

    rm $HOME/pip.log

    pip2.7 -v -v -v --log $HOME/pip.log install --user --build $HOME/tmp/pip mod_wsgi-httpd

Rather than unpack the package in /tmp somewhere, it will use a directory '$HOME/tmp/pip'.

If that fails with the same problem, what do you get for:

    ls -las $HOME/tmp/pip/mod-wsgi-httpd/build/apr-1.5.1/

Note that once pip is done, it doesn't appear to actually remove the directory $HOME/tmp/pip so you would need to remove it manually to reclaim the space.

Graham

[Paul Royik]

rm $HOME/pip.log

 pip2.7 -v -v -v --log $HOME/pip.log install --user --build $HOME/tmp/pip mod_wsgi-httpd

"Successfully installed mod-wsgi-httpd-2.4.12.3"

Next, mod_wsgi-apxs -q INCLUDEDIR outputs

"/home/username/lib/python2.7/mod_wsgi/httpd/include"

pip2.7 install --user mod_wsgi:

"Successfully installed mod-wsgi-4.4.7"

mod-wsgi-express

-bash: mod-wsgi-express: command not found

What next?

[Graham Dumpleton]

That one is simple, typo on my part. Should be:

mod_wsgi-express

and should output usage.

Remember to remove:

$HOME/tmp/pip

as it will take up over 100MB of space.

The install mod_wsgi-httpd and mod_wsgi packages should take less than 15MB so that amount of space shouldn't cause you problems.

I will now work on next set of steps based on the configuration you gave me before.

Is late here now, so likely only tomorrow. Will see if I have a little bit of time before I go to sleep though to start looking.

Thanks for persisting with me.

Graham

[Paul Royik]

Usage: mod_wsgi-express command [params]

Commands:

    install-module

    module-location

    setup-server

    start-server

mod_wsgi-express: error: No command was specified.

I removed tmp/bin

Thank you very much.

Will be waiting for your response.

[Graham Dumpleton]

BTW, what WebFaction is probably doing is:

    https://www.debian-administration.org/article/57/Making_/tmp_non-executable

That is mounting /tmp with 'noexec' mount option.

Good to have actually come across that, as would have hit it later in another way when running mod_wsgi-express in steps I will give you if didn't know.

Graham

[Paul Royik]

Glad that everything worked out, so we can proceed later.

[Paul Royik]

Hello.

Is there any way we can proceed?

Site is down.

[Graham Dumpleton]

On 04/02/2015, at 6:22 PM, Paul Royik <distan...@gmail.com> wrote:

> Hello.
> Is there any way we can proceed?

Before we do, I want to make the point that nothing of what we do could help.

There is an assumption here on my part that you have yourself at least done something to eliminate the possibility that it is your own application code which is causing the issue.

On the basis that you have and you do not believe it is your own code which is causing the problem, I can only steer you away from using Apache 2.2 which has known issues in a couple of areas.

These particular issues with Apache 2.2 in particular are as follows:

1. There is no flow control in the Apache child worker processes for response content.

What this means is that if a web application is generating a large response a bit at a time, if you have a slow HTTP client which isn't consuming that response quick enough, Apache will start to buffer up that response content in memory at a growing rate. It will not actually block if the amount of buffered response content exceeds some level and so stop itself from going back to the web application to continually get more.

This will affect a Python web application under mod_wsgi whether embedded mode or daemon mode is used. The memory growth is always in the Apache child worker process though. So in the case of daemon mode, it isn't actually the daemon process with your web application in it that sees growing memory usage, but the Apache worker child process that is proxying the request to the daemon process.

This issue in Apache 2.2 is exacerbated due to how memory pooling works in Apache, where each request thread has its own memory pool, but where there is no default upper bound on the size of that memory pool for each request thread. This means that once a slow HTTP client will cause the persistent memory pool for that one request thread to peak, but memory will not be released back for use by other parts of the application. In a multithread Apache worker, this can in turn happen for every request thread, and so the peak memory usage will occur for each thread in turn if it is common occurrence. So if each request thread peaked at 5MB memory usage and you had 25 threads, that will result in the worker process consuming at least 75MB just because of the lack of flow control.

Apache 2.4 fixes this issue and introduces flow control and will block when the accumulated response content exceeds 64Kb, and will not go back to the web application to get more until it has been able to write that buffered content. You therefore don't see this peaking of memory usage per request thread and memory usage stays flat.

Although Apache 2.4 fixes the lack of flow control, Apache 2.4 finally now has a threshold on memory pool size and at the end of a request, it will release back memory to the rest of the application for reuse if the threshold is exceeded.

To get around the flow control with Apache 2.2, you either have to use Apache 2.4, or use a more recent mod_wsgi version. A recent mod_wsgi helps as it implements its own flow control to get around problems in Apache 2.2.

Which ever is used, rather than seeing a worker process balloon out in size to 75MB, then it should stay down in the range of a few MB.

2. The older Apache 1.3 C APIs for reading request content will consume a bit of memory for each read, which isn't released until the end of the request back to the memory pool.

This means that if you are dealing with very large request content, and especially if you have a slow HTTP client with data trickling in, you will end up with lots of separate reads of the request content, with each causing a slow growth in memory usage because of memory not being reused immediately.

Because of older mod_wsgi versions supporting Apache 1.3, it was using this old API with this issue and so could be affected. With mod_wsgi 4.X support for Apache 1.3 was dropped and allowed a switch to be made to Apache 2.X APIs which didn't have this issue.

So this issue could actually technically effect either Apache 2.2 or Apache 2.4 and is only avoided by using a recent mod_wsgi version.

If running your Python web application in embedded mode the issue would be exacerbated by the memory pool thresholding issue as explained above.

If using daemon mode then the issue is occurring in the daemon mode process. In that case memory pools are disposed of at the end of a request and not reused across requests so issue perhaps not as significant, but still can be an issue.

Anyway, the point of explaining these in detail is so that you know and can make a judgment of whether you could even have been affected. If you were not consuming large uploads, nor generating large responses, then any memory issue is unlikely to be due to these. That is where you still need to consider whether it is your own application code.

That all said I will end this email, and start dealing with you need to do next in subsequent emails. That you keep cutting out the discussion history from the emails means I have to go search back through old emails to find all the details.

Graham

[Paul Royik]

I'm sure that my application does nothing that eats 600 mb of memory.

Also, I have a question. After we've made installations, site is constantly receiving 502 error. I didn't change anything. Is this possible?

[Graham Dumpleton]

On 04/02/2015, at 8:57 PM, Paul Royik <distan...@gmail.com> wrote:

> I'm sure that my application does nothing that eats 600 mb of memory.
>
> Also, I have a question. After we've made installations, site is constantly receiving 502 error. I didn't change anything. Is this possible?

When you say constantly, do you mean every single request fails and no request at all are being served up?

What error messages if any are in the Apache error log at:

/home/simamura/logs/user/error_django_app.log

I don't expect that what I have had you install should interfere in anyway with your existing Apache/mod_wsgi setup.

I would need to see any actual error messages from the error log to know what the issue is.

Graham

[Paul Royik]

In user/error.log there is "premature end of script: wsgi.py". This error resolves automatically after server restart.

In frontend/error.log "connect() failed (111: connection refused) while connecting to upstream"

[Graham Dumpleton]

On 04/02/2015, at 9:05 PM, Paul Royik <distan...@gmail.com> wrote:

> In user/error.log there is "premature end of script: wsgi.py". This error resolves automatically after server restart.

Can I see more than just that message, but also any messages from before and after, with time stamps included.

Basically give me the exact full messages spanning a few minutes either side of that.

I would also recommend you modify:

/home/simamura/webapps/django_app/apache2/conf/httpd.conf

and set:

LogLevel info

This will cause messages output by mod_wsgi about process restarts and the reason for them so can see what the trigger may have been.

BTW, some of the things you provide me before don't make too much sense when I look back at them.

So can you tell me what you get for:

ls -las $HOME/webapps/django_app

And also can you tell me what the name of your Django project directory is under 'django_app'?

You configuration refers to 'proj', but also 'mathsite'. Unless you have been renaming things in what you have sent me, the Apache configuration for the application doesn't appear to be consistent and doesn't look like it is actually picking up things from the correct places, although the WebFaction default setup looks a bit dodgy to start with.

Graham

[Paul Royik]

ls -las $HOME/webapps/django_app 

4 drwxr-xr-x  6 simamura simamura 4096 Jan 23 19:15 .

4 drwxr-xr-x 12 root     root     4096 Feb  3 09:34 ..

4 drwxr-xr-x  7 simamura simamura 4096 Jan 23 12:42 apache2

4 drwxr-xr-x  3 simamura simamura 4096 Jan 23 12:42 bin

4 drwxr-xr-x  3 simamura simamura 4096 Jan 23 12:42 lib

4 drwxr-xr-x  5 simamura simamura 4096 Jan 23 13:52 mathsite

I actually renamed everything to mathsite.

If you wish, I can give you access to webfaction account, so you can see it yourself.

[Graham Dumpleton]

On 04/02/2015, at 9:19 PM, Paul Royik <distan...@gmail.com> wrote:

> ls -las $HOME/webapps/django_app
>
> 4 drwxr-xr-x 6 simamura simamura 4096 Jan 23 19:15 .
> 4 drwxr-xr-x 12 root root 4096 Feb 3 09:34 ..
> 4 drwxr-xr-x 7 simamura simamura 4096 Jan 23 12:42 apache2
> 4 drwxr-xr-x 3 simamura simamura 4096 Jan 23 12:42 bin
> 4 drwxr-xr-x 3 simamura simamura 4096 Jan 23 12:42 lib
> 4 drwxr-xr-x 5 simamura simamura 4096 Jan 23 13:52 mathsite
>
> I actually renamed everything to math site.

What is your current httpd.conf you have then.

Before you gave:

WSGIDaemonProcess django_app processes=2 threads=12 python-path=/home/simamura/webapps/django_app:/home/simamura/webapps/django_app/mathsite:/home/simamura/webapps/django_app/lib/python2.7

WSGIScriptAlias / /home/simamura/webapps/django_app/proj/proj/wsgi.py

That can't be right though as the WSGIScriptAlias is not referring to 'math site' but 'proj'.

> If you wish, I can give you access to webfaction account, so you can see it yourself.

Rather not.

Graham

[Paul Royik]

OK. In reality I have webapp django_math and inside it there is django site mathsite:

ServerRoot "/home/simamura/webapps/django_math/apache2"

LoadModule dir_module        modules/mod_dir.so

LoadModule env_module        modules/mod_env.so

LoadModule log_config_module modules/mod_log_config.so

LoadModule mime_module       modules/mod_mime.so

LoadModule rewrite_module    modules/mod_rewrite.so

LoadModule setenvif_module   modules/mod_setenvif.so

LoadModule wsgi_module       modules/mod_wsgi.so

LoadModule alias_module      modules/mod_alias.so

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

CustomLog /home/simamura/logs/user/access_django_math.log combined

ErrorLog /home/simamura/logs/user/error_django_math.log

KeepAlive Off

Listen 20241

MaxSpareThreads 3

MinSpareThreads 1

ServerLimit 1

SetEnvIf X-Forwarded-SSL on HTTPS=1

ThreadsPerChild 5

WSGIDaemonProcess django_math processes=2 threads=5 python-path=/home/simamura/webapps/django_math:/home/simamura/webapps/django_math/mathsite:/home/simamura/webapps/django_math/lib/python2.7

WSGIProcessGroup django_math

WSGIRestrictEmbedded On

WSGILazyInitialization On

WSGIScriptAlias / /home/simamura/webapps/django_math/mathsite/mathsite/wsgi.py

The real change here compared to previous week (on previous week there were no errors) is that I set threads=5 (not 12) as you advised.

[Graham Dumpleton]

The error message:

    "premature end of script: wsgi.py"

in the version of mod_wsgi you are using would generally indicate that the daemon process had crashed in some way.

This is why I asked for the surrounding messages from the Apache error log as there would be other messages in there confirming whether the daemon process had in fact crashed. Using LogLevel of info would also results in extra messages which would confirm what was happening.

One cause for a crashing process is that you are using a Python module with a C extension which is not written so as to be safe to use in Python sub interpreters.

    http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API

When only running the one Python web application, you should really be forcing it to run in the main interpreter context and that will avoid the problem.

Thus add to your configuration:

    WSGIApplicationGroup %{GLOBAL}

Graham

[Paul Royik]

I actually have two webapplications django_math and django_site.

Do I still need to add WSGIApplicationGroup %{GLOBAL}?

[Paul Royik]

And want to add, while django_math crashed with 502 error, another application was working without problems.

[Graham Dumpleton]

On 04/02/2015, at 9:41 PM, Paul Royik <distan...@gmail.com> wrote:

> I actually have two webapplications django_math and django_site.
>
> Do I still need to add WSGIApplicationGroup %{GLOBAL}?

You are going to make it very confusing for me if you are moving/renaming things.

The original Apache configuration you gave me had:

/home/simamura/webapps/django_app

The latest had:

/home/simamura/webapps/django_math

and now you are saying you actually have one called django_site.

Whatever you now have, my understanding of WebFaction is that each has a separate directory with separate Apache configuration.

So you would be running up multiple Apache instances, each using its own port.

Thus you probably have:

/home/simamura/webapps/django_math/apache2/httpd.conf
/home/simamura/webapps/django_site/apache2/httpd.conf

The Listen directive in each will be different.

If they are separate in this way, you can quite happily add:

WSGIApplicationGroup %{GLOBAL}

to each of the Apache configuration files as the sites are running in completely different setups.

Did you ever provide me the output from running:

crontab -l

That will tell me exactly what sites you are running and where they are.

Graham

[Paul Royik]

Sorry to confuse you.

There are two webapps: django_gradis and django_math. They separately have apache2 folder. So I can safely add WSGIApplicationGroup %{GLOBAL}  to both?

crontab -l 

15,35,55 * * * * ~/webapps/django_math/apache2/bin/start

10,30,50 * * * * ~/webapps/django_gradis/apache2/bin/start

[Graham Dumpleton]

If they definitely only have the one WSGIScriptAlias directive in their respective Apache httpd.conf file then yes, you can add:

WSGIApplicationGroup %{GLOBAL}

to both.

I am not sure how long you have had these sites but I gave feedback to WebFaction a long time ago that they should be setting this directive, so either you have an old site, or they didn't pay attention to what I was saying.

Graham

[Paul Royik]

I order hosting less than 3 weeks ago. I'm still on 60day money-back.

And I really don't know what to do as problems start. My previous hosting bluehost didn't have such problems.

Anyway, what should I do next. Let's assume that there is no error 502 anymore.

[Graham Dumpleton]

One more question for now.

Do you actually access your sites via a https URL or only http.

WebFaction adds to the Apache configuration:

SetEnvIf X-Forwarded-SSL on HTTPS=1

If you aren't using https URLs then I don't have to worry about that with any instructions.

Graham

[Paul Royik]

Only http

[Graham Dumpleton]

Do you understand how your Django static file assets are being served up with WebFaction?

Have you set a static only WebFaction application in their control panel to serve static file assets for each of your sites and are already using 'python2.7 manage.py collectstatic' to copy them under a directory specified by STATIC_ROOT setting in Django settings file?

http://docs.webfaction.com/software/django/config.html#serving-django-static-media

Or do you believe that WebFaction is relying on django.contrib.staticfiles for serving up static file assets even though it isn't recommended for production?

Graham

[Paul Royik]

I  created separate application and serve files from there.

"Or do you believe that WebFaction is relying on django.contrib.staticfiles for serving up static file assets even though it isn't recommended for production? "   I don't know.

[Graham Dumpleton]

On 04/02/2015, at 10:52 PM, Paul Royik <distan...@gmail.com> wrote:

> I created separate application and serve files from there.
>
> "Or do you believe that WebFaction is relying on django.contrib.staticfiles for serving up static file assets even though it isn't recommended for production? " I don't know.

I am going to take that as meaning you followed steps in:

http://docs.webfaction.com/software/django/config.html#serving-django-static-media

It is past 11pm now and I don't want to be giving you the instructions right before I sleep so will continue this tomorrow night.

There is only one thing I am not 100% sure on right now and that is how Webfaction handles log rotation.

They say they rotate logs here:

http://docs.webfaction.com/software/general.html#accessing-logs

and they use logrotate to do that.

Normally logrotate is setup to send a signal to a web server to reload after the log file has been renamed.

I can't see how WebFaction is sending that signal though as they claim to rotate any log file in that directory no matter what creates it.

I can only assume they use the copytruncate option for logrotate to avoid sending a signal. That is, copy the log file and then truncate the original.

I will assume that is the case initially.

Anyway, I believe I have it all sorted as to what is going to be the easiest way to transition.

Before I outline that, I might suggest you have a play with mod_wsgi-express on your own personal system at home if you have Linux or MacOS X. By doing that you will have an idea what I intend to describe you switch to. Key thing is that the Apache configuration will be generated automatically, which is different to what you might be used to.

So have a look at:

https://pypi.python.org/pypi/mod_wsgi

If you don't have working Apache on your own system that 'pip install mod_wsgi' works against, you can do the 'pip install mod_wsgi-httpd' again and it will setup a usable Apache for you which will then be used.

This is all for your personal system for now and not on WebFaction.

For WebFaction there is going to be a bunch of options that will need to be specified to mod_wsgi-express when setting up a configuration to satisfy various requirements they have. No harm playing with it at home though first.

Graham

[Paul Royik]

OK.

[Paul Royik]

Completely forgot, but webfaction sent me a manual for installing new apache and mod_wsgi. Can you look at it and tell me your thoughts. Maybe it will help you.

Both apache and mod_wsgi have extensive documentation at:

 - http://httpd.apache.org/docs/2.4/install.html
 - https://code.google.com/p/modwsgi/wiki/QuickInstallationGuide

Please do look into them. If you do start using a custom Apache/mod_wsgi stack you will need to be able to configure and maintain it on your own, since it is outside the scope of our support.

The steps you need to take over an ssh session, according to the above documentation, to have a working Apache 2.4/mod_wsgi 4.4.6 stack is:

cd ~
mkdir src
cd src
wget http://mirrors.gigenet.com/apache/apr/apr-1.5.1.tar.gz
wget http://mirrors.gigenet.com/apache/apr/apr-util-1.5.4.tar.gz
wget http://mirrors.gigenet.com/apache/httpd/httpd-2.4.10.tar.gz
wget https://codeload.github.com/GrahamDumpleton/mod_wsgi/tar.gz/4.4.6
tar xzvf httpd-2.4.10.tar.gz
tar xvzf 4.4.6
cd httpd-2.4.10/srclib
tar xvzf ../../apr-1.5.1.tar.gz
tar xvzf ../../apr-util-1.5.4.tar.gz
mv apr-1.5.1 apr
mv apr-util-1.5.4 apr-util
cd ..
/configure --prefix=$HOME/webapps/<app>/apache24/ --with-included-apr
make
make install
cd ../mod_wsgi-4.4.6/
 ./configure --with-apxs=$HOME/webapps/<app>/apache24/bin/apxs --with-python=/usr/local/bin/python2.7
make
make install
cd $HOME/webapps/<app>/apache24/conf

Then change the Listen directive in the httpd.conf file to the port your application is going to use according to our control panel.

Also add the following line near the LoadModule stanza:

LoadModule wsgi_module modules/mod_wsgi.so

Other changes to the httpd.conf file might be needed, like removing unneeded modules that stack up memory, and set up django as per https://docs.djangoproject.com/en/1.7/howto/deployment/wsgi/modwsgi/

When that's done, you should be able to start apache using ~/webapps/<app>/apache24/bin/apachectl -k start.

[Paul Royik]

[Graham Dumpleton]

Would likely put you in a worse position as they absolve themselves from telling you anything about how to set it up properly.

So more than likely you would end up with something worse than what they had for other reasons.

Their default configuration for their installer avoids various issues, but using old httpd and mod_wsgi as they do is a big issue. What they should have done in your case was at least say to start with the Apache configuration you had from the initial setup. You would also need to swap crontab entries over as will explain later.

Using mod_wsgi-express you will at least get an Apache configuration generated automatically which is tailored for running a Python web application using mod_wsgi.

I have only just come online for the evening. I will start describing next steps in subsequent email.

Graham

[Paul Royik]

OK.

Thank you.

[Graham Dumpleton]

Take it slow. Don't rush.

I am awake for about another 45 minutes.

We are going to start with:

$HOME/webapps/django_math

because that is what you supplied the last Apache configuration file.

Read through all the steps below first. Only when you believe you
understand things then consider doing anything. If you don't understand
something then ask.

Be tolerant of minor mistakes and see if you can work out what I stuffed
up. For example if a path is wrong.

If when get to point where have stopped old site and trying to run new,
if there is an issue, then ensure new is stopped and restore things so
old runs again while work out what issue was.

The steps are:

1. Install a newer version of mod_wsgi than you currently have. This is a
development version which has a few new options we are going to need to
match how WebFaction handles logfiles. I added them specifically to support
your setup. :-)

To do this run:

pip2.7 install -U https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip

Test it is installed okay by running:

mod_wsgi-express

with no arguments. It should output the usage.

2. Create a file called:

$HOME/webapps/django_math/express-config

In it place:

#!/bin/sh

APPL_NAME=django_math
SITE_NAME=mathsite
SITE_PORT=20241

PYTHON_VERSION=2.7

SITE_ROOT=${HOME}/webapps/${APPL_NAME}

LOG_DIR=$HOME/logs/user
LOG_FORMAT="%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""

cd ${SITE_ROOT}

mod_wsgi-express setup-server \
--working-directory ${SITE_NAME} \
--server-root express \
--port ${SITE_PORT} \
--log-directory ${LOG_DIR} \
--access-log --access-log-format "${LOG_FORMAT}" \
--access-log-name access_${APPL_NAME}.log \
--error-log-name error_${APPL_NAME}.log \
--python-path ${SITE_ROOT}/lib/python${PYTHON_VERSION} \
$SITE_NAME/$SITE_NAME/wsgi.py

### End express-config.

3. Make the express-config script executable.

chmod +x express-config

This script is what is going to be used to generate the Apache configuration
for you and setup all the managements scripts for starting up and shutting
down Apache.

I have got you to create a script for it so the options you use are
recorded and you don't forget them. You may want to tweak the configuration
at some point later by adding more options. Such as to change number of
processes and threads or timeout values.

Once this is working, then you can duplicate the script to equivalent
location for your second application and change the APPL_NAME, SITE_NAME
and SITE_PORT settings to match and follow similar procedure.

4. Run the express-config script:

cd $HOME/webapps/django_math
./express-config

This should output something that looks like:

Server URL : http://localhost:20241/
Server Root : /home/simamura/webapps/django_math/express
Server Conf : /home/simamura/webapps/django_math/express/httpd.conf
Error Log File : /home/simamura/logs/user/error_django_math.log
Access Log File : /home/simamura/logs/user/access_django_math.log
Environ Variables : /home/simamura/webapps/django_math/express/envvars
Control Script : /home/simamura/webapps/django_math/express/apachectl
Request Capacity : 5 (1 process * 5 threads)
Request Timeout : 60 (seconds)
Queue Backlog : 100 (connections)
Queue Timeout : 45 (seconds)
Server Capacity : 20 (event/worker), 20 (prefork)
Server Backlog : 500 (connections)
Locale Setting : en_US.UTF-8

This has not run Apache at this point, it only generates the configuration.

To see what it has generated, look inside the 'express' subdirectory. That
is, as listed against 'Server Root'.

Look around in here to see what is generated, but never modify any files in
here. Well actually, it is possible to modify the 'envvars' file as that one
file will not be overridden if you run the script again, where as other
files in the directory will be, but right now I don't think we need to
modify 'envvars'.

In short, always modify express-config and rerun it to modify the
configuration, never edit generated configuration directly as you will
loose the changes if express-config is run again.

5. We now need to test this runs up your application okay.

Unfortunately WebFaction assigns specific port numbers to you to use and
so it is a bit hard to choose a random port just to test if this will
now start up. We therefore actually need to stop your existing site first
so we can run this.

The first step you need to do here is stop WebFaction from restarting
your existing site when you stop it.

To do this you need to edit the crontab file for your account and comment
out the cronjob entry for the site.

So first up run:

crontab -l

This will yield:

15,35,55 * * * * ~/webapps/django_math/apache2/bin/start
10,30,50 * * * * ~/webapps/django_gradis/apache2/bin/start

You are going to comment out the entry for 'django_math'.

To do that run:

crontab -e

This will throw you into 'vi', 'vim', or whatever else may be set as your
editor.

Edit the file so it looks like:

# 15,35,55 * * * * ~/webapps/django_math/apache2/bin/start

10,30,50 * * * * ~/webapps/django_gradis/apache2/bin/start

Save the file and exit the editor. This will stop cron from restarting
your existing Apache instance for 'django_math' when it is stopped.

6. Stop your existing Apache instance for 'django_math'.

Run:

$HOME/webapps/django_math/apache2/bin/stop

7. Start up the mod_wsgi-express generated version of the Apache
configuration.

This is done by running the 'apachectl' script listed under 'Control Script'
in what was generated by 'express-config'.

When run it MUST be given the 'start' argument.

Run:

$HOME/webapps/django_math/express/apachectl start

8. Attempt to access your web site using its external URL.

If that doesn't work, then we need to shutdown the new Apache instance
by running:

$HOME/webapps/django_math/express/apachectl stop

That is, same control script but with the 'stop' argument.

The restart your old Apache instance to get that running again.

$HOME/webapps/django_math/apache2/bin/start

Reinstate the controb entry, by running:

crontab -e

and restoring the file to:

15,35,55 * * * * ~/webapps/django_math/apache2/bin/start
10,30,50 * * * * ~/webapps/django_gradis/apache2/bin/start

Check with:

crontab -l

that it is back how it was.

Now look in:

$HOME/logs/user/error_django_math.log

and if it has any errors from when new Apache instance was started up
then send it to me so I can work out what went wrong.

Skip step 9.

9. If step 8 all worked fine and your site is working okay and all your
testing shows it is all perfect, including static file assets and you
want to now run it permanently, edit the crontab file again and add in
command to ensure that new Apache is started.

Run:

crontab -e

change it to read:

#15,35,55 * * * * ~/webapps/django_math/apache2/bin/start
15,35,55 * * * * ~/webapps/django_math/express/apachectl start

10,30,50 * * * * ~/webapps/django_gradis/apache2/bin/start

Thus, old command still commented out and new one which runs:

~/webapps/django_math/express/apachectl start

When run for a while, use 'ps auxwww' to look for process entries with names:

httpd (mod_wsgi-express)
(wsgi:localhost:20241:XXX)

where XXX is your UNIX account uid.

See how much memory they are using.

If there isn't a great deal of difference to before, then it is
questionable as to whether the memory issue is due to Apache. Would still
be interested in what memory sizes you are getting for processes.

[Paul Royik]

express-config  file should have what extension?

[Paul Royik]

I've got error from first step pip2.7 install -U https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip 

Collecting https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip

  Downloading https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip (675kB)

    100% |################################| 675kB 5.1MB/s

Installing collected packages: mod-wsgi

  Found existing installation: mod-wsgi 4.4.7

    Uninstalling mod-wsgi-4.4.7:

      Successfully uninstalled mod-wsgi-4.4.7

  Running setup.py install for mod-wsgi

    building 'mod_wsgi.server.mod_wsgi-py27' extension

    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I -I/usr/local/include/python2.7 -c src/server/wsgi_restrict.c -o build/temp.linux-x86_64-2.7/src/server/wsgi_restrict.o

    In file included from src/server/wsgi_restrict.h:24,

                     from src/server/wsgi_restrict.c:21:

    src/server/wsgi_python.h:24:20: error: Python.h: No such file or directory

    src/server/wsgi_python.h:27:2: error: #error Sorry, Python developer package does not appear to be installed.

    src/server/wsgi_python.h:31:2: error: #error Sorry, mod_wsgi requires at least Python 2.3.0 for Python 2.X.

    src/server/wsgi_python.h:39:2: error: #error Sorry, mod_wsgi requires that Python supporting thread.

    src/server/wsgi_python.h:42:26: error: structmember.h: No such file or directory

    src/server/wsgi_python.h:43:21: error: compile.h: No such file or directory

    src/server/wsgi_python.h:44:18: error: node.h: No such file or directory

    src/server/wsgi_python.h:45:20: error: osdefs.h: No such file or directory

    src/server/wsgi_python.h:46:25: error: frameobject.h: No such file or directory

    In file included from src/server/wsgi_restrict.c:21:

    src/server/wsgi_restrict.h:31: error: expected specifier-qualifier-list before Б─≤PyObject_HEADБ─≥

    src/server/wsgi_restrict.h:35: error: expected Б─≤=Б─≥, Б─≤,Б─≥, Б─≤;Б─≥, Б─≤asmБ─≥ or Б─≤__attribute__Б─≥ before Б─≤Restricted_TypeБ─≥

    src/server/wsgi_restrict.c:25: error: expected Б─≤=Б─≥, Б─≤,Б─≥, Б─≤;Б─≥, Б─≤asmБ─≥ or Б─≤__attribute__Б─≥ before Б─≤Restricted_TypeБ─≥

    src/server/wsgi_restrict.c: In function Б─≤newRestrictedObjectБ─≥:

    src/server/wsgi_restrict.c:31: warning: implicit declaration of function Б─≤PyObject_NewБ─≥

    src/server/wsgi_restrict.c:31: error: expected expression before Б─≤RestrictedObjectБ─≥

    src/server/wsgi_restrict.c:32: error: Б─≤NULLБ─≥ undeclared (first use in this function)

    src/server/wsgi_restrict.c:32: error: (Each undeclared identifier is reported only once

    src/server/wsgi_restrict.c:32: error: for each function it appears in.)

    src/server/wsgi_restrict.c:35: error: Б─≤RestrictedObjectБ─≥ has no member named Б─≤sБ─≥

    src/server/wsgi_restrict.c: In function Б─≤Restricted_deallocБ─≥:

    src/server/wsgi_restrict.c:42: warning: implicit declaration of function Б─≤PyObject_DelБ─≥

    src/server/wsgi_restrict.c: At top level:

    src/server/wsgi_restrict.c:45: error: expected Б─≤=Б─≥, Б─≤,Б─≥, Б─≤;Б─≥, Б─≤asmБ─≥ or Б─≤__attribute__Б─≥ before Б─≤*Б─≥ token

    src/server/wsgi_restrict.c:52: error: expected Б─≤=Б─≥, Б─≤,Б─≥, Б─≤;Б─≥, Б─≤asmБ─≥ or Б─≤__attribute__Б─≥ before Б─≤Restricted_TypeБ─≥

    error: command 'gcc' failed with exit status 1

    Complete output from command /usr/local/bin/python2.7 -c "import setuptools, tokenize;__file__='/tmp/pip-sbKbzx-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-LJTMIQ-record/install-record.txt --single-version-externally-managed --compile:

    running install

    running build

    running build_py

    creating build

    creating build/lib.linux-x86_64-2.7

    creating build/lib.linux-x86_64-2.7/mod_wsgi

    copying src/__init__.py -> build/lib.linux-x86_64-2.7/mod_wsgi

    creating build/lib.linux-x86_64-2.7/mod_wsgi/server

    copying src/server/apxs_config.py -> build/lib.linux-x86_64-2.7/mod_wsgi/server

    copying src/server/__init__.py -> build/lib.linux-x86_64-2.7/mod_wsgi/server

    creating build/lib.linux-x86_64-2.7/mod_wsgi/server/management

    copying src/server/management/__init__.py -> build/lib.linux-x86_64-2.7/mod_wsgi/server/management

    creating build/lib.linux-x86_64-2.7/mod_wsgi/server/management/commands

    copying src/server/management/commands/runmodwsgi.py -> build/lib.linux-x86_64-2.7/mod_wsgi/server/management/commands

    copying src/server/management/commands/__init__.py -> build/lib.linux-x86_64-2.7/mod_wsgi/server/management/commands

    creating build/lib.linux-x86_64-2.7/mod_wsgi/docs

    copying docs/_build/html/__init__.py -> build/lib.linux-x86_64-2.7/mod_wsgi/docs

    creating build/lib.linux-x86_64-2.7/mod_wsgi/images

    copying images/__init__.py -> build/lib.linux-x86_64-2.7/mod_wsgi/images

    copying images/snake-whiskey.jpg -> build/lib.linux-x86_64-2.7/mod_wsgi/images

    running build_ext

    building 'mod_wsgi.server.mod_wsgi-py27' extension

    creating build/temp.linux-x86_64-2.7

    creating build/temp.linux-x86_64-2.7/src

    creating build/temp.linux-x86_64-2.7/src/server

    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I -I/usr/local/include/python2.7 -c src/server/wsgi_restrict.c -o build/temp.linux-x86_64-2.7/src/server/wsgi_restrict.o

    In file included from src/server/wsgi_restrict.h:24,

                     from src/server/wsgi_restrict.c:21:

    src/server/wsgi_python.h:24:20: error: Python.h: No such file or directory

    src/server/wsgi_python.h:27:2: error: #error Sorry, Python developer package does not appear to be installed.

    src/server/wsgi_python.h:31:2: error: #error Sorry, mod_wsgi requires at least Python 2.3.0 for Python 2.X.

    src/server/wsgi_python.h:39:2: error: #error Sorry, mod_wsgi requires that Python supporting thread.

    src/server/wsgi_python.h:42:26: error: structmember.h: No such file or directory

    src/server/wsgi_python.h:43:21: error: compile.h: No such file or directory

    src/server/wsgi_python.h:44:18: error: node.h: No such file or directory

    src/server/wsgi_python.h:45:20: error: osdefs.h: No such file or directory

    src/server/wsgi_python.h:46:25: error: frameobject.h: No such file or directory

    In file included from src/server/wsgi_restrict.c:21:

    src/server/wsgi_restrict.h:31: error: expected specifier-qualifier-list before Б─≤PyObject_HEADБ─≥

    src/server/wsgi_restrict.h:35: error: expected Б─≤=Б─≥, Б─≤,Б─≥, Б─≤;Б─≥, Б─≤asmБ─≥ or Б─≤__attribute__Б─≥ before Б─≤Restricted_TypeБ─≥

    src/server/wsgi_restrict.c:25: error: expected Б─≤=Б─≥, Б─≤,Б─≥, Б─≤;Б─≥, Б─≤asmБ─≥ or Б─≤__attribute__Б─≥ before Б─≤Restricted_TypeБ─≥

    src/server/wsgi_restrict.c: In function Б─≤newRestrictedObjectБ─≥:

    src/server/wsgi_restrict.c:31: warning: implicit declaration of function Б─≤PyObject_NewБ─≥

    src/server/wsgi_restrict.c:31: error: expected expression before Б─≤RestrictedObjectБ─≥

    src/server/wsgi_restrict.c:32: error: Б─≤NULLБ─≥ undeclared (first use in this function)

    src/server/wsgi_restrict.c:32: error: (Each undeclared identifier is reported only once

    src/server/wsgi_restrict.c:32: error: for each function it appears in.)

    src/server/wsgi_restrict.c:35: error: Б─≤RestrictedObjectБ─≥ has no member named Б─≤sБ─≥

    src/server/wsgi_restrict.c: In function Б─≤Restricted_deallocБ─≥:

    src/server/wsgi_restrict.c:42: warning: implicit declaration of function Б─≤PyObject_DelБ─≥

    src/server/wsgi_restrict.c: At top level:

    src/server/wsgi_restrict.c:45: error: expected Б─≤=Б─≥, Б─≤,Б─≥, Б─≤;Б─≥, Б─≤asmБ─≥ or Б─≤__attribute__Б─≥ before Б─≤*Б─≥ token

    src/server/wsgi_restrict.c:52: error: expected Б─≤=Б─≥, Б─≤,Б─≥, Б─≤;Б─≥, Б─≤asmБ─≥ or Б─≤__attribute__Б─≥ before Б─≤Restricted_TypeБ─≥

    error: command 'gcc' failed with exit status 1

    ----------------------------------------

    Rolling back uninstall of mod-wsgi

    Command "/usr/local/bin/python2.7 -c "import setuptools, tokenize;__file__='/tmp/pip-sbKbzx-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-LJTMIQ-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-sbKbzx-build

[Paul Royik]

Hello.

Can we proceed?

[Graham Dumpleton]

What do you get when you run:

    mod_wsgi-apxs -q INCLUDEDIR 

    mod_wsgi-apxs -q CPPFLAGS

    mod_wsgi-apxs -q CFLAGS

    mod_wsgi-apxs -q EXTRA_INCLUDES

    mod_wsgi-apxs -q EXTRA_CPPFLAGS

    mod_wsgi-apxs -q EXTRA_CFLAGS

Graham

[Graham Dumpleton]

Also send me the contents of the file:

    $HOME/lib/python2.7/site-packages/mod_wsgi-httpd/mod_wsgi/httpd/build/config_vars.mk

If that file does exist because your Python install has installed it in a slightly different location, find where the config_vars.mk file is by running:

    find lib/python/2.7 -name config_vars.mk

Graham

[Paul Royik]

[simamura@web373 ~]$ mod_wsgi-apxs -q INCLUDEDIR

/home/simamura/lib/python2.7/mod_wsgi/httpd/include

[simamura@web373 ~]$     mod_wsgi-apxs -q CPPFLAGS

[simamura@web373 ~]$     mod_wsgi-apxs -q CFLAGS

[simamura@web373 ~]$     mod_wsgi-apxs -q EXTRA_INCLUDES

-I/home/simamura/lib/python2.7/mod_wsgi/httpd/include -I. -I/home/simamura/lib/python2.7/mod_wsgi/httpd/include/apr-1 -I/home/simamura/lib/python2.7/mod_wsgi/httpd/include

[simamura@web373 ~]$     mod_wsgi-apxs -q EXTRA_CPPFLAGS

-DLINUX -D_REENTRANT -D_GNU_SOURCE

[simamura@web373 ~]$     mod_wsgi-apxs -q EXTRA_CFLAGS

-g -O2 -pthread

exp_exec_prefix = ${mod_wsgi_httpd_prefix}

exp_bindir = ${mod_wsgi_httpd_prefix}/bin

exp_sbindir = ${mod_wsgi_httpd_prefix}/bin

exp_libdir = ${mod_wsgi_httpd_prefix}/lib

exp_libexecdir = ${mod_wsgi_httpd_prefix}/modules

exp_mandir = ${mod_wsgi_httpd_prefix}/man

exp_sysconfdir = ${mod_wsgi_httpd_prefix}/conf

exp_datadir = ${mod_wsgi_httpd_prefix}

exp_installbuilddir = ${mod_wsgi_httpd_prefix}/build

exp_errordir = ${mod_wsgi_httpd_prefix}/error

exp_iconsdir = ${mod_wsgi_httpd_prefix}/icons

exp_htdocsdir = ${mod_wsgi_httpd_prefix}/htdocs

exp_manualdir = ${mod_wsgi_httpd_prefix}/manual

exp_cgidir = ${mod_wsgi_httpd_prefix}/cgi-bin

exp_includedir = ${mod_wsgi_httpd_prefix}/include

exp_localstatedir = ${mod_wsgi_httpd_prefix}

exp_runtimedir = ${mod_wsgi_httpd_prefix}/logs

exp_logfiledir = ${mod_wsgi_httpd_prefix}/logs

exp_proxycachedir = ${mod_wsgi_httpd_prefix}/proxy

EGREP = /bin/grep -E

PCRE_LIBS = -L${mod_wsgi_httpd_prefix}/lib -lpcre

SHLTCFLAGS = -prefer-pic

LTCFLAGS = -prefer-non-pic -static

MKINSTALLDIRS = ${mod_wsgi_httpd_prefix}/build/mkdir.sh

INSTALL = $(LIBTOOL) --mode=install ${mod_wsgi_httpd_prefix}/build/install.sh -c

MATH_LIBS = -lm

CRYPT_LIBS = -lcrypt

DTRACE = true

PICFLAGS =

PILDFLAGS =

INSTALL_DSO = yes

ab_CFLAGS =

ab_LDFLAGS = -lssl -lcrypto -lrt -lcrypt -lpthread

NONPORTABLE_SUPPORT = checkgid fcgistarter

progname = httpd

OS = unix

SHLIBPATH_VAR = LD_LIBRARY_PATH

AP_BUILD_SRCLIB_DIRS =

AP_CLEAN_SRCLIB_DIRS =

bindir = ${exec_prefix}/bin

sbindir = ${exec_prefix}/bin

cgidir = ${datadir}/cgi-bin

logfiledir = ${localstatedir}/logs

exec_prefix = ${prefix}

datadir = ${prefix}

localstatedir = ${prefix}

mandir = ${prefix}/man

libdir = ${exec_prefix}/lib

libexecdir = ${exec_prefix}/modules

htdocsdir = ${datadir}/htdocs

manualdir = ${datadir}/manual

includedir = ${prefix}/include

errordir = ${datadir}/error

iconsdir = ${datadir}/icons

sysconfdir = ${prefix}/conf

installbuilddir = ${datadir}/build

runtimedir = ${localstatedir}/logs

proxycachedir = ${localstatedir}/proxy

other_targets =

progname = httpd

prefix = ${mod_wsgi_httpd_prefix}

AWK = gawk

CC = gcc -std=gnu99

CPP = gcc -E

CXX =

CPPFLAGS =

CFLAGS =

CXXFLAGS =

LTFLAGS = --silent

LDFLAGS =

LT_LDFLAGS =

SH_LDFLAGS =

LIBS =

DEFS =

INCLUDES =

NOTEST_CPPFLAGS =

NOTEST_CFLAGS =

NOTEST_CXXFLAGS =

NOTEST_LDFLAGS =

NOTEST_LIBS =

EXTRA_CPPFLAGS = -DLINUX -D_REENTRANT -D_GNU_SOURCE

EXTRA_CFLAGS = -g -O2 -pthread

EXTRA_CXXFLAGS =

EXTRA_LDFLAGS =

EXTRA_LIBS =

EXTRA_INCLUDES = -I$(includedir) -I. -I${mod_wsgi_httpd_prefix}/include/apr-1 -I${mod_wsgi_httpd_prefix}/include

INTERNAL_CPPFLAGS =

LIBTOOL = ${mod_wsgi_httpd_prefix}/build-1/libtool --silent

SHELL = /bin/sh

RSYNC = /usr/bin/rsync

SH_LIBS =

SH_LIBTOOL = $(LIBTOOL)

MK_IMPLIB =

MKDEP = $(CC) -MM

INSTALL_PROG_FLAGS =

ENABLED_DSO_MODULES = ,authn_file,authn_core,authz_host,authz_groupfile,authz_user,authz_core,access_compat,auth_basic,reqtimeout,filter,mime,log_config,env,headers,setenvif,version,mpm_event,unixd,status,autoindex,dir,alias

LOAD_ALL_MODULES = no

APR_BINDIR = ${mod_wsgi_httpd_prefix}/bin

APR_INCLUDEDIR = ${mod_wsgi_httpd_prefix}/include/apr-1

APR_VERSION = 1.5.1

APR_CONFIG = ${mod_wsgi_httpd_prefix}/bin/apr-1-config

APU_BINDIR = ${mod_wsgi_httpd_prefix}/bin

APU_INCLUDEDIR = ${mod_wsgi_httpd_prefix}/include/apr-1

APU_VERSION = 1.5.4

APU_CONFIG = ${mod_wsgi_httpd_prefix}/bin/apu-1-config

[Graham Dumpleton]

Run:

    pip uninstall mod_wsgi-httpd mod_wsgi

Say yes to each.

Reinstall both:

    pip install mod_wsgi-httpd

    pip install https://github.com/GrahamDumpleton/mod_wsgi/archive/develop.zip

When trying to upgrade the mod_wsgi package pip is uninstalling the old one but in doing so is breaking the mod_wsgi-httpd install somehow.

Graham

[Paul Royik]

Again error.

Failed to build APR. I guess I need to use tmp folder.

[Graham Dumpleton]

On 06/02/2015, at 10:45 PM, Paul Royik <distan...@gmail.com> wrote:

> Again error.
> Failed to build APR. I guess I need to use tmp folder.

Yep:

pip2.7 -v -v -v --log $HOME/pip.log install --user --build $HOME/tmp/pip mod_wsgi-httpd

and cleanup as before.

Graham

[Paul Royik]

It worked.

What next? The steps you've already specified?

[Graham Dumpleton]

On 06/02/2015, at 10:54 PM, Paul Royik <distan...@gmail.com> wrote:

> It worked.
> What next? The steps you've already specified?

Yes, follow the details instructions I previously gave.

All you have done so far is step 1. Now do step 2 onwards.

Make sure you read everything and you are confident you know how to back out of the changes when you get to point of shutting down existing Apache and trying to run new one. That is, that you believe you know how to put back your old Apache if need be.

Time to sleep.

Good luck.

Graham

[Paul Royik]

OK.

[Paul Royik]

Thank you very much.

However, I have a couple of questions.

1. New apache didn't work with memcache. I need to install memcache. Why is that? Previous version of apache worked without installation.

2. I googled and found that webfaction memory usage is done with command  ps -u simamura -o pid,rss,command | awk '{print $0}{sum+=$2} END {print "Total", sum/1024, "MB"}'. On previous version of apache average value was 550MB and now it is 220MB.

Is there anyway we could find out what exactly processes eat memory. I still don't know whether it is my application, and if it is where to search. Maybe it is single page, that does such things. You gave me command  'ps auxwww'. Here is it output, but I can't understand what it means:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

simamura  2170  0.0  0.0  51548  2304 ?        Ss   14:30   0:00 /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start

simamura  2185  0.0  0.2 636824 44608 ?        Sl   14:30   0:00 /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start

simamura  2186  0.0  0.2 639128 47784 ?        Sl   14:30   0:01 /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start

simamura  2188  0.0  0.0 387536  2668 ?        Sl   14:30   0:00 /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start

simamura  9731 13.8  1.1 1148500 193184 ?      Sl   16:14  14:10 (wsgi:localhost:20241:1092) -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

simamura 10987  0.0  0.0 108304  1876 pts/12   Ss   17:50   0:00 -bash

simamura 22390  0.0  0.0 110228  1152 pts/12   R+   17:56   0:00 ps auxwww

simamura 22495  0.0  0.0  66380  2976 ?        Ss   15:59   0:00 httpd (mod_wsgi-express)    -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

simamura 31149  0.0  0.0 661460  3524 ?        Sl   16:05   0:01 httpd (mod_wsgi-express)    -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

simamura 31992  0.0  0.0 726996  3588 ?        Sl   16:05   0:01 httpd (mod_wsgi-express)    -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

Again, thank you very much for your time and help.

[Graham Dumpleton]

On 07/02/2015, at 4:57 AM, Paul Royik <distan...@gmail.com> wrote:

> Thank you very much.
> However, I have a couple of questions.
> 1. New apache didn't work with memcache. I need to install memcache. Why is that? Previous version of apache worked without installation.

How did you install the memcache library previously and do you know where was it installed in your account? That is way back originally before even trying this new mod_wsgi version.

If you reinstalled it now to get it working, what command did you use to do so?

If you haven't reinstalled memcache to get it working, what was the actual error message you were seeing in the Apache error logs or elsewhere where you capture exception details?

I didn't add back in every Python module search path directory as based on the information you gave they weren't required.

The only one that may well have been necessary to add back in as well was:

$HOME/lib/python2.7

but then that is where mod_wsgi-express itself was being installed and since that was found when run, I took it to mean that WebFaction had fiddled their Python installation to look there and so it wasn't necessary.

Can you provide the output from:

ls -las $HOME/lib/python2.7

> 2. I googled and found that webfaction memory usage is done with command ps -u simamura -o pid,rss,command | awk '{print $0}{sum+=$2} END {print "Total", sum/1024, "MB"}'. On previous version of apache average value was 550MB and now it is 220MB.
> Is there anyway we could find out what exactly processes eat memory. I still don't know whether it is my application, and if it is where to search. Maybe it is single page, that does such things. You gave me command 'ps auxwww'. Here is it output, but I can't understand what it means:
>
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND

> simamura 9731 13.8 1.1 1148500 193184 ? Sl 16:14 14:10 (wsgi:localhost:20241:1092) -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

This is your actual WSGI application process. That is, the one in the form (wsgi:localhost:XXX:1092) where XXX is the port number for Apache for this application.

It is using resident memory size of 193184 kilobytes, that is, just under 200 megabytes.

> simamura 22495 0.0 0.0 66380 2976 ? Ss 15:59 0:00 httpd (mod_wsgi-express) -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

Of these latter three which show all the same, this is the Apache parent process. At least that is my assumption based on it being the lowest process ID and having smallest resident memory size.

It is taking less than 3 megabytes.

> simamura 31149 0.0 0.0 661460 3524 ? Sl 16:05 0:01 httpd (mod_wsgi-express) -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start
> simamura 31992 0.0 0.0 726996 3588 ? Sl 16:05 0:01 httpd (mod_wsgi-express) -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

These are the Apache child worker processes which actually accept requests initially. In your setup all they are doing is proxying requests through to the WSGI application process. If you weren't setting up WebFaction to serve your static assets using the front end nginx process, they also could have been setup to serve up the static assets.

Important thing to realise here. Your prior configuration was using processes=2 for WSGIDaemonProcess. That was actually wasteful because the Apache child worker configuration wasn't allocating enough capacity to even proxy that many requests.

The current configuration therefore is only creating one WSGI application process and therefore your WSGI application memory in total would have been halved straight away.

So the question is whether your figure of 550MB is across all processes, or that is what you were getting for a single process.

I did tell you back in the start to change WSGIDaemonProcess back to not create 2 processes. I can't recollect whether you did that or not.

if you didn't and that 550MB memory size you were talking about was across all processes, then the end result is about right, we simply halved overall memory usage by dropping down to one WSGI application process.

If that is the case, although this whole exercise has been extremely useful to me for various reasons, it may well have been a waste of your time because you could simply have removed 'processes=2' option from WSGIDaemonProcess and got the same result.

Graham

[Paul Royik]

1. I installed it with pyp2.7 python-memcache. Actually, I don't remember whether I installed it last time.

ls -las $HOME/lib/python2.7 

 4 drwx------ 21 simamura simamura  4096 Feb  6 16:05 .

 4 drwx------  7 simamura simamura  4096 Jan 22 14:35 ..

 4 drwxrwxr-x  5 simamura simamura  4096 Jan 24 17:05 crispy_forms

 4 drwxrwxr-x  2 simamura simamura  4096 Jan 24 17:05 django_crispy_forms-1.4.0-py2.7.egg-info

 4 -rw-rw-r--  1 simamura simamura   244 Jan 23 17:13 easy-install.pth

 4 drwxrwxr-x  7 simamura simamura  4096 Jan 26 12:55 guppy

 4 drwxrwxr-x  2 simamura simamura  4096 Jan 26 11:39 guppy-0.1.10-py2.7.egg-info

52 -rw-rw-r--  1 simamura simamura 52383 Feb  6 16:05 memcache.py

48 -rw-rw-r--  1 simamura simamura 47133 Feb  6 16:05 memcache.pyc

 4 drwxrwxr-x  7 simamura simamura  4096 Feb  6 11:54 mod_wsgi

 4 drwxrwxr-x  2 simamura simamura  4096 Feb  6 11:54 mod_wsgi-4.4.8-py2.7.egg-info

 4 drwxrwxr-x  2 simamura simamura  4096 Feb  6 11:48 mod_wsgi_httpd-2.4.12.3-py2.7.egg-info

 4 drwxrwxr-x  4 simamura simamura  4096 Jan 23 17:13 numpy-1.8.1-py2.7-linux-x86_64.egg

 0 -rw-------  1 root     root         0 Jan 22 14:35 .ph

 4 drwxrwxr-x  4 simamura simamura  4096 Jan 23 16:07 pip-6.0.6-py2.7.egg

 4 drwxrwxr-x  2 simamura simamura  4096 Feb  6 16:05 python_memcached-1.53-py2.7.egg-info

 4 drwxrwxr-x  3 simamura simamura  4096 Jan 24 17:05 pytz

 4 drwxrwxr-x  2 simamura simamura  4096 Jan 24 17:05 pytz-2014.10.dist-info

 4 drwxrwxr-x  3 simamura simamura  4096 Jan 24 17:06 requests

 4 drwxrwxr-x  2 simamura simamura  4096 Jan 24 17:06 requests-2.5.1.dist-info

 4 drwxrwxr-x  3 simamura simamura  4096 Jan 23 16:08 share

 4 drwxrwxr-x 37 simamura simamura  4096 Jan 23 16:08 sympy

 4 drwxrwxr-x  2 simamura simamura  4096 Jan 23 16:08 sympy-0.7.6-py2.7.egg-info

12 drwxrwxr-x  2 simamura simamura 12288 Jan 24 17:08 unidecode

 4 drwxrwxr-x  2 simamura simamura  4096 Jan 24 17:08 Unidecode-0.04.17-py2.7.egg-info

"I did tell you back in the start to change WSGIDaemonProcess back to not create 2 processes. I can't recollect whether you did that or not. " Can you tell me where to watch and change it?

Also, I encountered 504 error. Is it natural or something went wrong? 

[Graham Dumpleton]

How much traffic does your site actually get and how long do the requests run for.

In express-config change it to:

mod_wsgi-express setup-server \
--working-directory ${SITE_NAME} \
--server-root express \
--port ${SITE_PORT} \
--log-directory ${LOG_DIR} \
--access-log --access-log-format "${LOG_FORMAT}" \
--access-log-name access_${APPL_NAME}.log \
--error-log-name error_${APPL_NAME}.log \
--python-path ${SITE_ROOT}/lib/python${PYTHON_VERSION} \

--threads 10 \
$SITE_NAME/$SITE_NAME/wsgi.py

That is, add the --threads option with 10. This will increase number of threads from 5 to 10 and increase WSGI application capacity.

The configuration for the Apache child workers will automatically increase to accommodate being able to proxy that many.

In short, the 504 means a gateway timeout, which can be due to overloading WSGI application capacity. I would need to see the actual error message from the Apache error log to know the scenario which is triggering it.

That the Apache child worker process no longer have a constrained capacity, requests will queue up in a different spot and so this may arise.

Depending on how long your requests run, we may need to tweak some of the timeouts.

Knowing the messages from Apache error log will tell me what is going on.

Graham

[Paul Royik]

In fact traffic is not very heavy.

3000 pageviews per day.

"how long do the requests run for" don't know. How to measure that? I have one page, that can run up to 150 sec. before timeout. Do I need to lower?

Below is the part of error log:

[Fri Feb 06 12:39:06.288100 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 12:39:06.288103 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 12:39:06.288107 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 12:39:11.254220 2015] [wsgi:info] [pid 9731:tid 140177922582272] mod_wsgi (pid=9731): Aborting process 'localhost:20241'.

[Fri Feb 06 12:39:11.254246 2015] [wsgi:info] [pid 9731:tid 140177922582272] mod_wsgi (pid=9731): Exiting process 'localhost:20241'.

[Fri Feb 06 18:39:11.286257 2015] [wsgi:error] [pid 31992:tid 140179135010560] [client 127.0.0.1:43870] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 18:39:11.286363 2015] [wsgi:error] [pid 31149:tid 140179069642496] [client 127.0.0.1:43747] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 18:39:11.286447 2015] [wsgi:error] [pid 31149:tid 140179135543040] [client 127.0.0.1:43796] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: https://www.google.com/

[Fri Feb 06 18:39:12.131163 2015] [wsgi:info] [pid 22495:tid 140179136616192] mod_wsgi (pid=9731): Process 'localhost:20241' has died, deregister and restart it.

[Fri Feb 06 18:39:12.131193 2015] [wsgi:info] [pid 22495:tid 140179136616192] mod_wsgi (pid=9731): Process 'localhost:20241' has been deregistered and will no longer be monitored.

[Fri Feb 06 18:39:12.131470 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Starting process 'localhost:20241' with threads=5.

[Fri Feb 06 18:39:12.131613 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Python home /usr/local.

[Fri Feb 06 18:39:12.131626 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Initializing Python.

[Fri Feb 06 18:39:12.184325 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Attach interpreter ''.

[Fri Feb 06 18:39:12.207432 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Imported 'mod_wsgi'.

[Graham Dumpleton]

Can you go back to before that traceback at start appears? Capture about a dozen lines before the stack frame lines started.

It does look like the request timeout.

Change express-config to have:

mod_wsgi-express setup-server \
--working-directory ${SITE_NAME} \
--server-root express \
--port ${SITE_PORT} \
--log-directory ${LOG_DIR} \
--access-log --access-log-format "${LOG_FORMAT}" \
--access-log-name access_${APPL_NAME}.log \
--error-log-name error_${APPL_NAME}.log \
--python-path ${SITE_ROOT}/lib/python${PYTHON_VERSION} \

--request-timeout 300 \
$SITE_NAME/$SITE_NAME/wsgi.py

So have added the --request-timeout option, increasing it from nominal 60 seconds to 300 seconds.

It isn't a hard 60 seconds because is a multithreaded process and that changes how it works so is actually variable. Too complicated to explain right now.

After you do make changes to express-config, remember to rerun it.

You can then run:

$HOME/webapps/django_math/express/apachectl restart

to have Apache reread the configuration.

Graham

[Paul Royik]

Changed and restarted.

More of error log:

[Fri Feb 06 10:14:27.479708 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/core/expr.py", line 158, in __div__,

[Fri Feb 06 10:14:27.479710 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/core/decorators.py", line 108, in binary_op_wrapper,

[Fri Feb 06 10:14:27.479713 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/core/decorators.py", line 70, in __sympifyit_wrapper,

[Fri Feb 06 10:14:27.479715 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/core/expr.py", line 1822, in extract_multiplicatively,

[Fri Feb 06 10:14:27.479718 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/core/expr.py", line 2033, in could_extract_minus_sign,

[Fri Feb 06 10:14:27.479720 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/simplify/cse_opts.py", line 12, in sub_pre,

[Fri Feb 06 10:14:27.479722 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 3482, in signsimp,

[Fri Feb 06 10:14:27.479725 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/core/basic.py", line 1429, in match,

[Fri Feb 06 10:14:27.479727 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_rewrites.py", line 5, in trig_rewrites,

[Fri Feb 06 10:14:27.479730 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479732 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479734 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479737 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 140, in <lambda>,

[Fri Feb 06 10:14:27.479739 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 138, in add_rule,

[Fri Feb 06 10:14:27.479742 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 94, in switch_rl,

[Fri Feb 06 10:14:27.479744 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479746 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479749 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479751 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 159, in substitution_rule,

[Fri Feb 06 10:14:27.479756 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_intergals.py", line 309, in tan_sec_power_tanodd,

[Fri Feb 06 10:14:27.479759 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/utils.py", line 9, in multiplexer_rl,

[Fri Feb 06 10:14:27.479762 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_intergals.py", line 37, in tansec_power_rule,

[Fri Feb 06 10:14:27.479765 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479767 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479770 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_intergals.py", line 90, in trig_powers_products_rule,

[Fri Feb 06 10:14:27.479773 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479775 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479777 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479780 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 140, in <lambda>,

[Fri Feb 06 10:14:27.479782 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 138, in add_rule,

[Fri Feb 06 10:14:27.479784 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 94, in switch_rl,

[Fri Feb 06 10:14:27.479787 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479789 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479792 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479794 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_intergals.py", line 309, in tan_sec_power_tanodd,

[Fri Feb 06 10:14:27.479797 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/utils.py", line 9, in multiplexer_rl,

[Fri Feb 06 10:14:27.479799 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_intergals.py", line 37, in tansec_power_rule,

[Fri Feb 06 10:14:27.479801 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479806 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479808 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_intergals.py", line 90, in trig_powers_products_rule,

[Fri Feb 06 10:14:27.479811 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479813 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479816 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479818 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 146, in mul_rule,

[Fri Feb 06 10:14:27.479821 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479823 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479825 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479828 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479830 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 140, in <lambda>,

[Fri Feb 06 10:14:27.479832 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 138, in add_rule,

[Fri Feb 06 10:14:27.479835 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 94, in switch_rl,

[Fri Feb 06 10:14:27.479837 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479839 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479842 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479844 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/utils.py", line 152, in _rewriter,

[Fri Feb 06 10:14:27.479847 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 32, in conditioned_rl,

[Fri Feb 06 10:14:27.479849 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479854 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 75, in try_rl,

[Fri Feb 06 10:14:27.479857 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479859 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479861 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479864 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.479867 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 10:14:27.479869 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 247, in _parts_rule,

[Fri Feb 06 10:14:27.479872 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 305, in parts_rule,

[Fri Feb 06 10:14:27.479874 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479877 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479879 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479881 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479884 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 146, in mul_rule,

[Fri Feb 06 10:14:27.479886 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479889 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479891 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479893 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479896 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.479899 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 10:14:27.479904 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479908 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 490, in print_Rewrite,

[Fri Feb 06 10:14:27.479910 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479913 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.479915 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.479917 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 10:14:27.479920 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479922 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 490, in print_Rewrite,

[Fri Feb 06 10:14:27.479925 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479927 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 10:14:27.479930 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479932 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 261, in print_Add,

[Fri Feb 06 10:14:27.479935 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479937 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 490, in print_Rewrite,

[Fri Feb 06 10:14:27.479940 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479942 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.479944 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 10:14:27.479947 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 247, in _parts_rule,

[Fri Feb 06 10:14:27.479951 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 305, in parts_rule,

[Fri Feb 06 10:14:27.479954 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479956 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479958 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479962 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479964 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 146, in mul_rule,

[Fri Feb 06 10:14:27.479967 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 10:14:27.479969 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479971 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 10:14:27.479974 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 10:14:27.479976 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.479979 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 10:14:27.479981 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479984 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 10:14:27.479986 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479988 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.479991 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 10:14:27.479993 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.479996 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 10:14:27.480001 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.480003 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.480006 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 10:14:27.480008 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.480011 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.480013 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 10:14:27.480015 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 10:14:27.480018 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 10:14:27.480020 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 10:14:27.480023 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/calculator36.py", line 14, in solve,

[Fri Feb 06 10:14:27.480025 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 385, in decor,

[Fri Feb 06 10:14:27.480028 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 10:14:27.480030 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 10:14:27.480032 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 10:14:27.480035 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): Thread 140178826864384 executing file "/usr/local/lib/python2.7/threading.py", line 309, in wait

[Fri Feb 06 10:14:27.480037 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/usr/local/lib/python2.7/threading.py", line 911, in join,

[Fri Feb 06 10:14:27.480040 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 345, in internal2,

[Fri Feb 06 10:14:27.480042 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 342, in calculator,

[Fri Feb 06 10:14:27.480045 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/base.py", line 78, in get_response,

[Fri Feb 06 10:14:27.480049 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/wsgi.py", line 228, in __call__,

[Fri Feb 06 10:14:27.480052 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1137, in handle_request.

[Fri Feb 06 10:14:27.480054 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): Thread 140178470991616 executing file "/home/simamura/lib/python2.7/sympy/polys/domains/expressiondomain.py", line 25, in __init__

[Fri Feb 06 10:14:27.480057 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/polys/domains/expressiondomain.py", line 58, in _to_ex,

[Fri Feb 06 10:14:27.480060 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/polys/domains/expressiondomain.py", line 86, in __mul__,

[Fri Feb 06 10:14:27.480062 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/matrices/matrices.py", line 2703, in <lambda>,

[Fri Feb 06 10:14:27.480065 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/matrices/dense.py", line 730, in zip_row_op,

[Fri Feb 06 10:14:27.480068 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/matrices/matrices.py", line 2659, in rref,

[Fri Feb 06 10:14:27.480070 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/polys/solvers.py", line 22, in solve_lin_sys,

[Fri Feb 06 10:14:27.480073 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 493, in _integrate,

[Fri Feb 06 10:14:27.480075 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 170, in heurisch,

[Fri Feb 06 10:14:27.480078 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 97, in heurisch_wrapper,

[Fri Feb 06 10:14:27.480080 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 628, in _eval_integral,

[Fri Feb 06 10:14:27.480083 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 358, in doit,

[Fri Feb 06 10:14:27.480085 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 1074, in integrate,

[Fri Feb 06 10:14:27.480088 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/utilities/decorator.py", line 17, in threaded_func,

[Fri Feb 06 10:14:27.480090 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/lib/python2.7/sympy/core/expr.py", line 2912, in integrate,

[Fri Feb 06 10:14:27.480092 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/calculator36.py", line 14, in solve,

[Fri Feb 06 10:14:27.480095 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 385, in decor,

[Fri Feb 06 10:14:27.480099 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 10:14:27.480102 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 10:14:27.480104 2015] [wsgi:info] [pid 31148:tid 140179136616192] mod_wsgi (pid=31148): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 10:14:32.464650 2015] [wsgi:info] [pid 31148:tid 140178418542336] mod_wsgi (pid=31148): Aborting process 'localhost:20241'.

[Fri Feb 06 10:14:32.464681 2015] [wsgi:info] [pid 31148:tid 140178418542336] mod_wsgi (pid=31148): Exiting process 'localhost:20241'.

[Fri Feb 06 16:14:32.476343 2015] [wsgi:error] [pid 31992:tid 140179070174976] [client 127.0.0.1:44740] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.google.com/url?sa=t&rct=j&q=antiderivative%20calculator%20step%20by%20step&source=web&cd=7&sqi=2&ved=0CEsQFjAG&url=http%3A%2F%2Fwww.emathhelp.net%2Fcalculators%2Fcalculus-2%2Fintegral-calculator%2F&ei=febUVJaOFYSoyQTHx4DgBA&usg=AFQjCNGPgXduyYwEzE020fX4uD0IKonVWw&surl=1&safe=active

[Fri Feb 06 16:14:32.476356 2015] [wsgi:error] [pid 31149:tid 140179135543040] [client 127.0.0.1:44873] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 16:14:32.476608 2015] [wsgi:error] [pid 31149:tid 140179136075520] [client 127.0.0.1:44714] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 16:14:32.476345 2015] [wsgi:error] [pid 31992:tid 140179136075520] [client 127.0.0.1:44778] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 16:14:33.282318 2015] [wsgi:info] [pid 22495:tid 140179136616192] mod_wsgi (pid=31148): Process 'localhost:20241' has died, deregister and restart it.

[Fri Feb 06 16:14:33.282354 2015] [wsgi:info] [pid 22495:tid 140179136616192] mod_wsgi (pid=31148): Process 'localhost:20241' has been deregistered and will no longer be monitored.

[Fri Feb 06 16:14:33.282630 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Starting process 'localhost:20241' with threads=5.

[Fri Feb 06 16:14:33.282807 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Python home /usr/local.

[Fri Feb 06 16:14:33.282825 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Initializing Python.

[Fri Feb 06 16:14:33.292991 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Attach interpreter ''.

[Fri Feb 06 16:14:33.293865 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Imported 'mod_wsgi'.

[Fri Feb 06 16:14:33.293917 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731, process='localhost:20241', application=''): Loading WSGI script '/home/simamura/webapps/django_math/express/handler.wsgi'.

[Fri Feb 06 16:16:28.386320 2015] [mpm_event:error] [pid 22495:tid 140179136616192] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

[Fri Feb 06 10:17:15.520369 2015] [wsgi:error] [pid 9731:tid 140178837354240] [remote 127.0.0.1:232] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 10:17:15.584752 2015] [wsgi:error] [pid 9731:tid 140178847844096] [remote 127.0.0.1:228] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 10:17:15.699458 2015] [wsgi:error] [pid 9731:tid 140178605733632] [remote 127.0.0.1:216] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 10:17:23.994859 2015] [wsgi:error] [pid 9731:tid 140178826864384] [remote 127.0.0.1:232] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:28.273759 2015] [wsgi:error] [pid 31149:tid 140179069642496] [client 127.0.0.1:48156] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 10:17:28.273978 2015] [wsgi:error] [pid 9731:tid 140178837354240] [remote 127.0.0.1:232] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:28.375990 2015] [wsgi:error] [pid 31992:tid 140179135543040] [client 127.0.0.1:48160] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 10:17:28.376252 2015] [wsgi:error] [pid 9731:tid 140178847844096] [remote 127.0.0.1:228] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:28.507389 2015] [wsgi:error] [pid 31149:tid 140179069376256] [client 127.0.0.1:48165] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 10:17:28.507603 2015] [wsgi:error] [pid 9731:tid 140178605733632] [remote 127.0.0.1:216] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:32.662105 2015] [wsgi:error] [pid 31992:tid 140179135010560] [client 127.0.0.1:48215] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 10:17:32.662339 2015] [wsgi:error] [pid 9731:tid 140178826864384] [remote 127.0.0.1:232] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 10:17:34.692362 2015] [wsgi:error] [pid 9731:tid 140178858333952] [remote 127.0.0.1:228] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:35.431299 2015] [wsgi:error] [pid 31149:tid 140179136341760] [client 127.0.0.1:48324] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 10:17:35.431537 2015] [wsgi:error] [pid 9731:tid 140178837354240] [remote 127.0.0.1:232] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:39.879326 2015] [wsgi:error] [pid 31149:tid 140179135543040] [client 127.0.0.1:48414] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 10:17:39.879544 2015] [wsgi:error] [pid 9731:tid 140178847844096] [remote 127.0.0.1:228] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:40.564004 2015] [wsgi:error] [pid 31992:tid 140179136341760] [client 127.0.0.1:48429] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/notes

[Fri Feb 06 10:17:40.564249 2015] [wsgi:error] [pid 9731:tid 140178605733632] [remote 127.0.0.1:216] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:43.400992 2015] [wsgi:error] [pid 31992:tid 140179069908736] [client 127.0.0.1:48500] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 10:17:43.401238 2015] [wsgi:error] [pid 9731:tid 140178826864384] [remote 127.0.0.1:232] mod_wsgi (pid=9731): Queue timeout expired for WSGI daemon process 'localhost:20241'.

[Fri Feb 06 16:17:43.519250 2015] [wsgi:error] [pid 31992:tid 140179069642496] [client 127.0.0.1:48501] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 16:17:44.142978 2015] [wsgi:error] [pid 31992:tid 140179069376256] [client 127.0.0.1:48512] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 16:17:54.226032 2015] [wsgi:error] [pid 31992:tid 140179070174976] [client 127.0.0.1:48541] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 16:17:54.433976 2015] [wsgi:error] [pid 31149:tid 140179135276800] [client 127.0.0.1:48555] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=ln%28x%29%2Fx%5E2%2Bx%5E2+cos%28x%29&var=&steps=on

[Fri Feb 06 16:17:55.168967 2015] [wsgi:error] [pid 31149:tid 140179136075520] [client 127.0.0.1:48800] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/questions

[Fri Feb 06 16:50:25.050208 2015] [wsgi:error] [pid 31149:tid 140179135276800] [client 127.0.0.1:60962] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/differential-equations/inverse-laplace-transform-calculator/

[Fri Feb 06 18:17:59.776008 2015] [wsgi:error] [pid 31992:tid 140179135010560] [client 127.0.0.1:48140] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=x%5E2%2F%287-x%5E3%29&var=&steps=on

[Fri Feb 06 18:18:17.265098 2015] [wsgi:error] [pid 31992:tid 140179069908736] [client 127.0.0.1:48460] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators

[Fri Feb 06 18:21:05.991047 2015] [wsgi:error] [pid 31149:tid 140179069642496] [client 127.0.0.1:51800] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators

[Fri Feb 06 18:23:38.438035 2015] [wsgi:error] [pid 31149:tid 140179135276800] [client 127.0.0.1:54660] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators

[Fri Feb 06 18:25:20.388096 2015] [wsgi:error] [pid 31992:tid 140179135010560] [client 127.0.0.1:56719] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/?f=x%5E2%2F%287-x%5E3%29&var=&steps=on

[Fri Feb 06 18:37:46.885102 2015] [wsgi:error] [pid 31149:tid 140179135543040] [client 127.0.0.1:41829] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 18:38:27.766527 2015] [wsgi:error] [pid 31992:tid 140179135276800] [client 127.0.0.1:42464] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators

[Fri Feb 06 18:38:38.967397 2015] [wsgi:error] [pid 31992:tid 140179070174976] [client 127.0.0.1:42646] Timeout when reading response headers from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 12:39:06.254056 2015] [wsgi:info] [pid 9731:tid 140178879313664] mod_wsgi (pid=9731): Daemon process request time limit exceeded, stopping process 'localhost:20241'.

[Fri Feb 06 12:39:06.254108 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Shutdown requested 'localhost:20241'.

[Fri Feb 06 12:39:06.286339 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Dumping stack trace for active Python threads.

[Fri Feb 06 12:39:06.286359 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178316318464 executing file "/usr/local/lib/python2.7/threading.py", line 147, in acquire

[Fri Feb 06 12:39:06.286363 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/compatibility.py", line 851, in wrapper,

[Fri Feb 06 12:39:06.286366 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/cache.py", line 89, in wrapper,

[Fri Feb 06 12:39:06.286369 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/operations.py", line 127, in _matches_commutative,

[Fri Feb 06 12:39:06.286372 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/mul.py", line 808, in matches,

[Fri Feb 06 12:39:06.286375 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/add.py", line 358, in _matches_simple,

[Fri Feb 06 12:39:06.286378 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/operations.py", line 127, in _matches_commutative,

[Fri Feb 06 12:39:06.286381 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/add.py", line 365, in matches,

[Fri Feb 06 12:39:06.286384 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/operations.py", line 127, in _matches_commutative,

[Fri Feb 06 12:39:06.286387 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/add.py", line 365, in matches,

[Fri Feb 06 12:39:06.286390 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/power.py", line 1021, in matches,

[Fri Feb 06 12:39:06.286393 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/operations.py", line 127, in _matches_commutative,

[Fri Feb 06 12:39:06.286396 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/mul.py", line 808, in matches,

[Fri Feb 06 12:39:06.286408 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/basic.py", line 1429, in match,

[Fri Feb 06 12:39:06.286411 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_rewrites.py", line 5, in trig_rewrites,

[Fri Feb 06 12:39:06.286415 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286417 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286420 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286423 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286426 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 12:39:06.286429 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/calculator36.py", line 14, in solve,

[Fri Feb 06 12:39:06.286432 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 385, in decor,

[Fri Feb 06 12:39:06.286435 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 12:39:06.286437 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 12:39:06.286440 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 12:39:06.286443 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140177934644992 executing file "/usr/local/lib/python2.7/threading.py", line 147, in acquire

[Fri Feb 06 12:39:06.286446 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/compatibility.py", line 851, in wrapper,

[Fri Feb 06 12:39:06.286448 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/cache.py", line 89, in wrapper,

[Fri Feb 06 12:39:06.286451 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/expr.py", line 158, in __div__,

[Fri Feb 06 12:39:06.286453 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/decorators.py", line 108, in binary_op_wrapper,

[Fri Feb 06 12:39:06.286456 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/decorators.py", line 70, in __sympifyit_wrapper,

[Fri Feb 06 12:39:06.286459 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 2452, in powsimp,

[Fri Feb 06 12:39:06.286462 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 2348, in powdenest,

[Fri Feb 06 12:39:06.286467 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/mul.py", line 1206, in breakup,

[Fri Feb 06 12:39:06.286470 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/mul.py", line 1183, in _eval_subs,

[Fri Feb 06 12:39:06.286473 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/basic.py", line 897, in _subs,

[Fri Feb 06 12:39:06.286475 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/compatibility.py", line 851, in wrapper,

[Fri Feb 06 12:39:06.286478 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/cache.py", line 89, in wrapper,

[Fri Feb 06 12:39:06.286481 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/basic.py", line 705, in subs,

[Fri Feb 06 12:39:06.286483 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_intergals.py", line 65, in cotcsc_power_rule,

[Fri Feb 06 12:39:06.286486 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286489 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286491 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_intergals.py", line 90, in trig_powers_products_rule,

[Fri Feb 06 12:39:06.286494 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286496 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286499 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286501 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 140, in <lambda>,

[Fri Feb 06 12:39:06.286504 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 138, in add_rule,

[Fri Feb 06 12:39:06.286507 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 94, in switch_rl,

[Fri Feb 06 12:39:06.286509 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286512 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286514 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286516 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286521 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 12:39:06.286524 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/calculator36.py", line 14, in solve,

[Fri Feb 06 12:39:06.286526 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 385, in decor,

[Fri Feb 06 12:39:06.286529 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 12:39:06.286531 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 12:39:06.286534 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 12:39:06.286536 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178295338752 executing file "/usr/local/lib/python2.7/threading.py", line 147, in acquire

[Fri Feb 06 12:39:06.286539 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/compatibility.py", line 851, in wrapper,

[Fri Feb 06 12:39:06.286541 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/cache.py", line 89, in wrapper,

[Fri Feb 06 12:39:06.286544 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/expr.py", line 143, in __rmul__,

[Fri Feb 06 12:39:06.286546 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/decorators.py", line 108, in binary_op_wrapper,

[Fri Feb 06 12:39:06.286549 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/decorators.py", line 70, in __sympifyit_wrapper,

[Fri Feb 06 12:39:06.286552 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 148, in collect,

[Fri Feb 06 12:39:06.286554 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 148, in collect,

[Fri Feb 06 12:39:06.286557 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/operations.py", line 127, in _matches_commutative,

[Fri Feb 06 12:39:06.286559 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/add.py", line 365, in matches,

[Fri Feb 06 12:39:06.286561 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/operations.py", line 127, in _matches_commutative,

[Fri Feb 06 12:39:06.286564 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/mul.py", line 808, in matches,

[Fri Feb 06 12:39:06.286566 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/basic.py", line 1429, in match,

[Fri Feb 06 12:39:06.286569 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/trig_rewrites.py", line 5, in trig_rewrites,

[Fri Feb 06 12:39:06.286573 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286576 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286578 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286581 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 159, in substitution_rule,

[Fri Feb 06 12:39:06.286584 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286586 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286589 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286591 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286593 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 146, in mul_rule,

[Fri Feb 06 12:39:06.286596 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286598 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286601 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286603 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286606 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286608 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 12:39:06.286611 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 247, in _parts_rule,

[Fri Feb 06 12:39:06.286614 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 305, in parts_rule,

[Fri Feb 06 12:39:06.286616 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286619 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286621 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286631 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286634 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 146, in mul_rule,

[Fri Feb 06 12:39:06.286636 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286639 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286642 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286644 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286647 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 140, in <lambda>,

[Fri Feb 06 12:39:06.286649 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 138, in add_rule,

[Fri Feb 06 12:39:06.286652 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 94, in switch_rl,

[Fri Feb 06 12:39:06.286654 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286656 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286659 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286661 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286664 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 12:39:06.286666 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 247, in _parts_rule,

[Fri Feb 06 12:39:06.286669 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 305, in parts_rule,

[Fri Feb 06 12:39:06.286671 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286674 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286676 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286681 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286683 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 146, in mul_rule,

[Fri Feb 06 12:39:06.286686 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.286688 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286691 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.286693 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.286696 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286699 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286702 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286704 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286707 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286709 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286712 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286714 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286717 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286719 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286722 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286724 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286731 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286736 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286740 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286744 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286749 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286753 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286757 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286762 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286766 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286770 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286775 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286779 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286783 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286788 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286792 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286797 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286801 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286806 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286814 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286819 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286824 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286829 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286834 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286838 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286843 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286848 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286853 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286858 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286863 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286867 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286869 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286872 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286874 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286877 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286879 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286882 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286887 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286890 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286892 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286895 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286897 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286900 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286902 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286905 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286907 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286910 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286912 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286915 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286917 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286920 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286922 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286924 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286927 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286929 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286934 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286937 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286940 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286943 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286945 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286947 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286950 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286952 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286955 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286957 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286960 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286962 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286964 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286967 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286969 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286972 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286974 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286977 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286981 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286984 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286986 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.286989 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286991 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.286994 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.286996 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.286998 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287001 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287003 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287006 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287009 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287011 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287014 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287016 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287019 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287021 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287026 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287029 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287031 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287034 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287036 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287038 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287041 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287043 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287046 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287048 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287051 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287053 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287055 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287058 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287060 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287063 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287065 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287068 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287072 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287075 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287078 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287081 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287083 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287086 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287088 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287091 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287093 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287095 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287098 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287100 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287103 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287105 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287108 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287110 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287112 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287115 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287121 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287124 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287127 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287129 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287131 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287134 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287136 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287139 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287141 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287144 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287146 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287148 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287151 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287154 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287158 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287163 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287167 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287172 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287180 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287185 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287189 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287193 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287197 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287206 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287210 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287215 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287219 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287224 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287229 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287234 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287239 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287243 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287247 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287251 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287256 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287261 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287269 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287274 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287279 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287284 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287289 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287294 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287299 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287303 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287308 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287313 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287317 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287322 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287327 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287332 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287335 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287338 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287340 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287346 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287349 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287351 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287354 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287356 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287358 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287361 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287363 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287366 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287368 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287370 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287373 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287375 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287378 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287380 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287383 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287385 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287388 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287392 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287395 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287397 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287400 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287402 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287405 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287407 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287409 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287412 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287414 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287417 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287419 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287421 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287424 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287426 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287429 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287431 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287434 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287438 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287440 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287443 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287446 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287448 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287450 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287453 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287455 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287458 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287460 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287463 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287465 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287468 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287470 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287473 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287475 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287478 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287480 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287485 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287487 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287490 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287492 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287495 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287497 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287500 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287502 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287505 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287507 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287510 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287512 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287514 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287517 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287519 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287522 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287524 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287527 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287531 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287534 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287536 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287539 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287541 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287544 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287546 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287549 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287551 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287553 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287556 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287558 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 210, in print_ConstantTimes,

[Fri Feb 06 12:39:06.287561 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287563 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287566 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 337, in print_Parts,

[Fri Feb 06 12:39:06.287568 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 147, in print_rule,

[Fri Feb 06 12:39:06.287571 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.287575 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 12:39:06.287578 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/calculator36.py", line 14, in solve,

[Fri Feb 06 12:39:06.287580 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 385, in decor,

[Fri Feb 06 12:39:06.287583 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 12:39:06.287585 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 12:39:06.287588 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 12:39:06.287591 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178847844096 executing file "/usr/local/lib/python2.7/threading.py", line 309, in wait

[Fri Feb 06 12:39:06.287594 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 911, in join,

[Fri Feb 06 12:39:06.287598 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 345, in internal2,

[Fri Feb 06 12:39:06.287603 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 342, in calculator,

[Fri Feb 06 12:39:06.287607 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/base.py", line 78, in get_response,

[Fri Feb 06 12:39:06.287612 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/wsgi.py", line 228, in __call__,

[Fri Feb 06 12:39:06.287617 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1137, in handle_request.

[Fri Feb 06 12:39:06.287622 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178284062464 executing file "/home/simamura/lib/python2.7/sympy/matrices/dense.py", line 730, in zip_row_op

[Fri Feb 06 12:39:06.287628 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/matrices/matrices.py", line 2659, in rref,

[Fri Feb 06 12:39:06.287632 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/polys/solvers.py", line 22, in solve_lin_sys,

[Fri Feb 06 12:39:06.287637 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 493, in _integrate,

[Fri Feb 06 12:39:06.287642 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 170, in heurisch,

[Fri Feb 06 12:39:06.287647 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 97, in heurisch_wrapper,

[Fri Feb 06 12:39:06.287652 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 628, in _eval_integral,

[Fri Feb 06 12:39:06.287657 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 358, in doit,

[Fri Feb 06 12:39:06.287666 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 1074, in integrate,

[Fri Feb 06 12:39:06.287671 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/utilities/decorator.py", line 17, in threaded_func,

[Fri Feb 06 12:39:06.287676 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/expr.py", line 2912, in integrate,

[Fri Feb 06 12:39:06.287681 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/calculator36.py", line 14, in solve,

[Fri Feb 06 12:39:06.287686 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 385, in decor,

[Fri Feb 06 12:39:06.287690 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 12:39:06.287695 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 12:39:06.287700 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 12:39:06.287704 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178858333952 executing file "/usr/local/lib/python2.7/threading.py", line 309, in wait

[Fri Feb 06 12:39:06.287708 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 911, in join,

[Fri Feb 06 12:39:06.287713 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 345, in internal2,

[Fri Feb 06 12:39:06.287716 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 342, in calculator,

[Fri Feb 06 12:39:06.287721 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/base.py", line 78, in get_response,

[Fri Feb 06 12:39:06.287725 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/wsgi.py", line 228, in __call__,

[Fri Feb 06 12:39:06.287729 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1137, in handle_request.

[Fri Feb 06 12:39:06.287734 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178326808320 executing file "/home/simamura/lib/python2.7/sympy/polys/fields.py", line 254, in __nonzero__

[Fri Feb 06 12:39:06.287739 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/polys/fields.py", line 349, in __sub__,

[Fri Feb 06 12:39:06.287743 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/matrices/matrices.py", line 2703, in <lambda>,

[Fri Feb 06 12:39:06.287747 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/matrices/dense.py", line 730, in zip_row_op,

[Fri Feb 06 12:39:06.287751 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/matrices/matrices.py", line 2659, in rref,

[Fri Feb 06 12:39:06.287760 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/polys/solvers.py", line 22, in solve_lin_sys,

[Fri Feb 06 12:39:06.287764 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 493, in _integrate,

[Fri Feb 06 12:39:06.287769 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 170, in heurisch,

[Fri Feb 06 12:39:06.287773 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 170, in heurisch,

[Fri Feb 06 12:39:06.287777 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 170, in heurisch,

[Fri Feb 06 12:39:06.287782 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 170, in heurisch,

[Fri Feb 06 12:39:06.287786 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 170, in heurisch,

[Fri Feb 06 12:39:06.287790 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/heurisch.py", line 97, in heurisch_wrapper,

[Fri Feb 06 12:39:06.287794 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 628, in _eval_integral,

[Fri Feb 06 12:39:06.287799 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 358, in doit,

[Fri Feb 06 12:39:06.287803 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/integrals/integrals.py", line 1074, in integrate,

[Fri Feb 06 12:39:06.287808 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/utilities/decorator.py", line 17, in threaded_func,

[Fri Feb 06 12:39:06.287812 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/expr.py", line 2912, in integrate,

[Fri Feb 06 12:39:06.287816 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/calculator36.py", line 14, in solve,

[Fri Feb 06 12:39:06.287820 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 385, in decor,

[Fri Feb 06 12:39:06.287824 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 12:39:06.287827 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 12:39:06.287831 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 12:39:06.287836 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178837354240 executing file "/usr/local/lib/python2.7/threading.py", line 309, in wait

[Fri Feb 06 12:39:06.287840 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 911, in join,

[Fri Feb 06 12:39:06.287844 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 345, in internal2,

[Fri Feb 06 12:39:06.287851 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 342, in calculator,

[Fri Feb 06 12:39:06.287855 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/base.py", line 78, in get_response,

[Fri Feb 06 12:39:06.287859 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/wsgi.py", line 228, in __call__,

[Fri Feb 06 12:39:06.287863 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1137, in handle_request.

[Fri Feb 06 12:39:06.287868 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178605733632 executing file "/usr/local/lib/python2.7/threading.py", line 309, in wait

[Fri Feb 06 12:39:06.287872 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 911, in join,

[Fri Feb 06 12:39:06.287876 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 345, in internal2,

[Fri Feb 06 12:39:06.287880 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 342, in calculator,

[Fri Feb 06 12:39:06.287884 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/base.py", line 78, in get_response,

[Fri Feb 06 12:39:06.287889 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/wsgi.py", line 228, in __call__,

[Fri Feb 06 12:39:06.287894 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1137, in handle_request.

[Fri Feb 06 12:39:06.287898 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178826864384 executing file "/usr/local/lib/python2.7/threading.py", line 309, in wait

[Fri Feb 06 12:39:06.287903 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 911, in join,

[Fri Feb 06 12:39:06.287907 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 345, in internal2,

[Fri Feb 06 12:39:06.287911 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 342, in calculator,

[Fri Feb 06 12:39:06.287916 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/base.py", line 78, in get_response,

[Fri Feb 06 12:39:06.287920 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/lib/python2.7/django/core/handlers/wsgi.py", line 228, in __call__,

[Fri Feb 06 12:39:06.287924 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1137, in handle_request.

[Fri Feb 06 12:39:06.287929 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): Thread 140178305828608 executing file "/usr/local/lib/python2.7/threading.py", line 147, in acquire

[Fri Feb 06 12:39:06.287933 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/compatibility.py", line 851, in wrapper,

[Fri Feb 06 12:39:06.287942 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/cache.py", line 89, in wrapper,

[Fri Feb 06 12:39:06.287946 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/mul.py", line 677, in as_real_imag,

[Fri Feb 06 12:39:06.287951 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/add.py", line 662, in as_real_imag,

[Fri Feb 06 12:39:06.287956 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/function.py", line 385, in _should_evalf,

[Fri Feb 06 12:39:06.287960 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/function.py", line 379, in <genexpr>,

[Fri Feb 06 12:39:06.287965 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/function.py", line 352, in __new__,

[Fri Feb 06 12:39:06.287969 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/compatibility.py", line 851, in wrapper,

[Fri Feb 06 12:39:06.287973 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/core/cache.py", line 89, in wrapper,

[Fri Feb 06 12:39:06.287978 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/fu.py", line 606, in f,

[Fri Feb 06 12:39:06.287983 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 4071, in bottom_up,

[Fri Feb 06 12:39:06.287987 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 4071, in bottom_up,

[Fri Feb 06 12:39:06.287991 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/fu.py", line 589, in TR8,

[Fri Feb 06 12:39:06.287995 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/fu.py", line 606, in f,

[Fri Feb 06 12:39:06.288000 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 4071, in bottom_up,

[Fri Feb 06 12:39:06.288003 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/fu.py", line 589, in TR8,

[Fri Feb 06 12:39:06.288007 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 117, in minrule,

[Fri Feb 06 12:39:06.288012 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 43, in chain_rl,

[Fri Feb 06 12:39:06.288017 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 117, in minrule,

[Fri Feb 06 12:39:06.288021 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 4319, in _futrig,

[Fri Feb 06 12:39:06.288026 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 4276, in futrig,

[Fri Feb 06 12:39:06.288031 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 1419, in <lambda>,

[Fri Feb 06 12:39:06.288040 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 1341, in trigsimp,

[Fri Feb 06 12:39:06.288045 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/simplify/simplify.py", line 3546, in simplify,

[Fri Feb 06 12:39:06.288049 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/utils.py", line 19, in test_subterm,

[Fri Feb 06 12:39:06.288053 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/utils.py", line 16, in find_substitutions,

[Fri Feb 06 12:39:06.288058 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 159, in substitution_rule,

[Fri Feb 06 12:39:06.288063 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 65, in null_safe_rl,

[Fri Feb 06 12:39:06.288067 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.288072 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/lib/python2.7/sympy/strategies/core.py", line 84, in do_one_rl,

[Fri Feb 06 12:39:06.288077 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/rules.py", line 411, in integral_steps,

[Fri Feb 06 12:39:06.288082 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 112, in _integrate,

[Fri Feb 06 12:39:06.288087 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/integral/printer.py", line 107, in integrate,

[Fri Feb 06 12:39:06.288091 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/calculators/calculator36.py", line 14, in solve,

[Fri Feb 06 12:39:06.288096 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/views/content.py", line 385, in decor,

[Fri Feb 06 12:39:06.288100 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/home/simamura/webapps/django_math/mathsite/mathsite/utils.py", line 352, in run,

[Fri Feb 06 12:39:06.288103 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 792, in __bootstrap_inner,

[Fri Feb 06 12:39:06.288107 2015] [wsgi:info] [pid 9731:tid 140179136616192] mod_wsgi (pid=9731): called from file "/usr/local/lib/python2.7/threading.py", line 769, in __bootstrap.

[Fri Feb 06 12:39:11.254220 2015] [wsgi:info] [pid 9731:tid 140177922582272] mod_wsgi (pid=9731): Aborting process 'localhost:20241'.

[Fri Feb 06 12:39:11.254246 2015] [wsgi:info] [pid 9731:tid 140177922582272] mod_wsgi (pid=9731): Exiting process 'localhost:20241'.

[Fri Feb 06 18:39:11.286257 2015] [wsgi:error] [pid 31992:tid 140179135010560] [client 127.0.0.1:43870] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 18:39:11.286363 2015] [wsgi:error] [pid 31149:tid 140179069642496] [client 127.0.0.1:43747] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: http://www.emathhelp.net/calculators/calculus-2/integral-calculator/

[Fri Feb 06 18:39:11.286447 2015] [wsgi:error] [pid 31149:tid 140179135543040] [client 127.0.0.1:43796] Truncated or oversized response headers received from daemon process 'localhost:20241': /home/simamura/webapps/django_math/express/htdocs/calculators, referer: https://www.google.com/

[Fri Feb 06 18:39:12.131163 2015] [wsgi:info] [pid 22495:tid 140179136616192] mod_wsgi (pid=9731): Process 'localhost:20241' has died, deregister and restart it.

[Fri Feb 06 18:39:12.131193 2015] [wsgi:info] [pid 22495:tid 140179136616192] mod_wsgi (pid=9731): Process 'localhost:20241' has been deregistered and will no longer be monitored.

[Fri Feb 06 18:39:12.131470 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Starting process 'localhost:20241' with threads=5.

[Fri Feb 06 18:39:12.131613 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Python home /usr/local.

[Fri Feb 06 18:39:12.131626 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Initializing Python.

[Fri Feb 06 18:39:12.184325 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Attach interpreter ''.

[Fri Feb 06 18:39:12.207432 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495): Imported 'mod_wsgi'.

[Fri Feb 06 18:39:12.207500 2015] [wsgi:info] [pid 12495:tid 140179136616192] mod_wsgi (pid=12495, process='localhost:20241', application=''): Loading WSGI script '/home/simamura/webapps/django_math/express/handler.wsgi'.

[Fri Feb 06 19:44:45.843190 2015] [core:error] [pid 31149:tid 140179135276800] (36)File name too long: [client 127.0.0.1:59741] AH00036: access to /properties-and-graph-of-the-function-y-cos-x+y=cosx \xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd/ failed (filesystem path '/home/simamura/webapps/django_math/express/htdocs/properties-and-graph-of-the-function-y-cos-x+y=cosx \xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd\xc3\x83\xc6\x92\xc3\x82\xc2\xaf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbf\xc3\x83\xe2\x80\x9a\xc3\x82\xc2\xbd')

[Paul Royik]

Hello.

Did error clear the situation?

[Paul Royik]

Hello.

What is the difference between request-timeout and socket-timeout?

[Graham Dumpleton]

On 09/02/2015, at 9:00 PM, Paul Royik <distan...@gmail.com> wrote:

> Hello.
> What is the difference between request-timeout and socket-timeout?

Sorry, have had some dramas to sort out. Explaining this will have to wait until tomorrow.

What exactly is your application doing anyway that it has request response times that can exceed well over 1 minute?

It is not generally regarded as good practice to be doing long duration processing in process to a web server. It is generally better to farm it off to a backend task execution system and for the web request to lodge the task and then use some manner of polling mechanism to check for completion.

Graham

[Paul Royik]

One page on site is calculating integral.

There is time limit of 150 sec. If 150 seconds passed and no solution, then Python raises an Exception and request is returned (no solution).

What is the best way to cut memory? Reduce time? Will this really help?

Will 60 or 90 seconds reduce memory?

What will be consequences if there will be more load on the page (more visitors)?

Thank you. Will be waiting for your answer.

Paul.

[Graham Dumpleton]

> What is the difference between request-timeout and socket-timeout?

Socket timeout is a very low level timeout value that relates to how long a blocking operation on a socket connection should allow to go on before timing out.

If using mod_wsgi directly and not via mod_wsgi-express, the socket-timeout option to WSGIDaemonProcess directive, if not defined, will default to the same value as is set for Apache as a whole using the Apache Timeout directive. For Apache 2.2 the Apache Timeout directive defaults to 300 seconds. In Apache 2.4, this has been reduced down to 60 seconds.

If using mod_wsgi-express the socket-timeout option is defaulted to 60 seconds and doesn't matter whether using Apache 2.2 or 2.4.

For mod_wsgi the socket-timeout relates specifically to communication over the socket connection to the daemon processes.

Before I explain what request-timeout is for, if you are now seeing 504 gateway timeout errors and logged messages of:

[Tue Feb 10 22:00:50 2015] [error] [client ::1] Timeout when reading response headers from daemon process 'localhost:8000': /tmp/mod_wsgi-localhost:8000:502/htdocs/

we will need to adjust this. The fact that your requests are long running but do not actually return any response content until done complicates things and means the default needs to be set higher in line with changes to request-timeout.

One important point with socket-timeout is that it will cause the Apache worker child process to timeout on waiting for a response from the daemon process. Although it will timeout and return a 504 gateway timeout error, it doesn't actually stop the request which would still be running in the daemon process.

Now we get to request-timeout. This is a fail safe timeout to protect against hung requests causing all available request threads to be consumed and for your daemon process running your web application to stop.

If using mod_wsgi directly, no request-timeout is specified at all and so there is no protection against hanging requests.

If using mod_wsgi-express, the request-timeout is 60 seconds.

If the daemon process were single threaded, what will happen is that any HTTP client would see the socket timeout expire and so it would see a 504 gateway timeout.

At the same time, within the daemon process the request-timeout would expire and the daemon process will enter an automatic restart to forcibly terminate the long running request on basis that since has reach 60 seconds, it has got stuck.

Remember at this point that web applications would want to generally return with seconds and preferably sub second. A web application is not generally the best place to be running very long running operations.

That said, the default timeouts are biased to what would be the norm for web applications, which is very fast requests.

Now, if the daemon process is multithreaded things get complicated.

This is because if one request only were to reach request-timeout, but there were much younger requests which had only just started running, you don't really want to go restarting the process as you will interrupt the requests which just started.

The best solution I could come up with to handle this situation of a multithread daemon is that when the total running time of all currently active requests, divided by the number of threads for the process, reaches the request-timeout only then would the process be forcibly restarted. In effect the notional average running time based on capacity.

This means that if only the one request was running and you had default of 5 threads, if it had been running for 60 seconds, then in fact you get a value of 12, which isn't greater than 60, so nothing has done.

So because there was spare capacity, the request is actually allowed to keep running much longer than it would otherwise as there is no harm doing so since we still have 4 spare threads with capacity to handle new requests.

If no other requests came in, then that one request could actually run out to 300 seconds before the process was forcibly restarted.

If however you had five requests start at the same time and they all ran out to 60 seconds, the derived value would be 60 and a restart would happen straight away.

So in a multithread process an attempt is made to allow requests to keep running while it looks like there is still capacity available.

As to the log messages you gave me, it is a bit hard for me to relate this to what you saw as I don't know what the number of threads was you were running. It seems you had increased it beyond 5. And I didn't know what the request-timeout was set to either.

What one can see is that the socket-timeout did trigger too early.

> One page on site is calculating integral.
> There is time limit of 150 sec. If 150 seconds passed and no solution, then Python raises an Exception and request is returned (no solution).

If you have a hard limit within your algorithm of 150 seconds, and since you do not return any data as the request goes on, change the options in express-config to have:

--request-timeout 180
--socket-timeout 180

The --socket-timeout option is new.

I can't comment really on what the --threads option should be at this point.

> What is the best way to cut memory? Reduce time? Will this really help?

I don't know what your algorithm for integral calculation does.

Does the algorithm itself cause a lot of memory use?

Have you tried extracting the integral algorithm into a standalone Python script separate from your web application and run it and watched how much memory it uses?

Does the memory usage grow the longer it runs, or does it plateau and stay flat?

If it keeps growing in memory usage and so the only thing that keeps down the memory usage is how long it runs for, you probably need to look at how your algorithm works and whether it is working in an efficient manner such that memory can be relaxed and reused as it goes.

> Will 60 or 90 seconds reduce memory?

No idea. Depends on what your integral algorithm is doing.

> What will be consequences if there will be more load on the page (more visitors)?

If the problem is your algorithm, having more threads to handle the load will cause greater overall memory usage.

How complicated is this algorithm? Are you using a package which has implemented such algorithms in an efficient way or have you implemented it yourself?

Graham

[Paul Royik]

Thank you for your response.

I asked about socket-timeout, because yesterday I cahnged it to 170 and left request-timeout on 60 seconds.

And it worked. There was no 504 error.

I tried simultaneously on different computers access calculator and it showed no drastic increase in memory usage. Same 200-250 MB level.

I use packaged algorithm (efficient) with slight modifications.

"Have you tried extracting the integral algorithm into a standalone Python script separate from your web application and run it and watched how much memory it uses? " I don't know this area of Python. How can I watch memory usage of piece of code?

"As to the log messages you gave me, it is a bit hard for me to relate this to what you saw as I don't know what the number of threads was you were running. It seems you had increased it beyond 5. And I didn't know what the request-timeout was set to either." you told me to set number of threads to 10. request-timeout was 300 (then I change it, but log was for 300 seconds).

Thank you.

[Graham Dumpleton]

On 10/02/2015, at 10:51 PM, Paul Royik <distan...@gmail.com> wrote:

Thank you for your response.

I asked about socket-timeout, because yesterday I cahnged it to 170 and left request-timeout on 60 seconds.

And it worked. There was no 504 error.

I tried simultaneously on different computers access calculator and it showed no drastic increase in memory usage. Same 200-250 MB level.

I use packaged algorithm (efficient) with slight modifications.

"Have you tried extracting the integral algorithm into a standalone Python script separate from your web application and run it and watched how much memory it uses? " I don't know this area of Python. How can I watch memory usage of piece of code?

Time to sleep, but have a look at the 'ps' command for a start.

Using 'ps auxwww' will show columns for memory use. The manual page (man ps) will explain what you are seeing.

"As to the log messages you gave me, it is a bit hard for me to relate this to what you saw as I don't know what the number of threads was you were running. It seems you had increased it beyond 5. And I didn't know what the request-timeout was set to either." you told me to set number of threads to 10. request-timeout was 300 (then I change it, but log was for 300 seconds).

Graham

[Paul Royik]

Output of  ps auxwww is

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

simamura  3282  0.0  0.0 108304  1888 pts/12   Ss+  12:44   0:00 -bash

simamura  7115  0.0  0.0  51548  1972 ?        Ss   Feb09   0:01 /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start

simamura  7116  0.0  0.2 636824 39208 ?        Sl   Feb09   0:02 /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start

simamura  7117  0.0  0.2 636824 44344 ?        Sl   Feb09   0:02 /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start

simamura  7118  0.0  0.0 322000  2484 ?        Sl   Feb09   0:00 /home/simamura/webapps/django_gradis/apache2/bin/httpd.worker -f /home/simamura/webapps/django_gradis/apache2/conf/httpd.conf -k start

simamura 14365  0.0  0.0  66380  2216 ?        Ss   Feb09   0:01 httpd (mod_wsgi-express)    -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

simamura 14372  0.0  0.0 858588  3972 ?        Sl   Feb09   0:12 httpd (mod_wsgi-express)    -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

simamura 15442  0.0  0.0 108304  1892 pts/13   Ss   12:52   0:00 -bash

simamura 16341  5.0  1.1 991380 186968 ?       Sl   11:50   3:18 (wsgi:localhost:20241:1092) -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

simamura 17572  0.0  0.0 793052  4128 ?        Sl   Feb09   0:11 httpd (mod_wsgi-express)    -f /home/simamura/webapps/django_math/express/httpd.conf -DWSGI_ACCESS_LOG -DWSGI_MPM_ENABLE_EVENT_MODULE -DWSGI_MPM_EXISTS_EVENT_MODULE -DWSGI_MPM_EXISTS_WORKER_MODULE -DWSGI_MPM_EXISTS_PREFORK_MODULE -k start

simamura 19510  0.0  0.0 110236  1160 pts/13   R+   12:55   0:00 ps auxwww

But it still unclear is it calculator page or some other pages. Is there anyway to measure memory usage of calulcator/page in isolation?

Thank you.

[Graham Dumpleton]

That is why I said to separate out your algorithm for your calculator into a separate runnable command line Python script such that you can give it different input and duration to run. As you run it in one window from the command line, use ps in another window to watch whether the RSS memory value keeps increasing over time. That way yo can know whether it is an issue or not and will eliminate that key algorithm as being the source of your major memory usage.

If it uses practically no memory, then it is going to be something else in your actual web application which is the issue.

Graham

[Paul Royik]

I used Python memory_profiler.

In the worst case (algorithm run 150 sec.) memory usage increase from 40MB to 47MB.

What will you say?

[Paul Royik]

Hi again.

Just got 502 error.

logs:

[Wed Feb 11 03:39:29.207365 2015] [wsgi:info] [pid 20794:tid 140288199374592] mod_wsgi (pid=20794): Shutdown requested 'localhost:20241'.

[Wed Feb 11 03:39:29.218543 2015] [wsgi:info] [pid 20794:tid 140288199374592] mod_wsgi (pid=20794): Stopping process 'localhost:20241'.

[Wed Feb 11 09:39:29.219070 2015] [wsgi:info] [pid 20787:tid 140288199374592] mod_wsgi (pid=20794): Process 'localhost:20241' to be deregistered, as server is restarting or being shutdown.

[Wed Feb 11 09:39:29.237292 2015] [wsgi:info] [pid 20787:tid 140288199374592] mod_wsgi (pid=20794): Process 'localhost:20241' has been deregistered and will no longer be monitored.

[Wed Feb 11 03:39:29.238325 2015] [wsgi:info] [pid 20794:tid 140288199374592] mod_wsgi (pid=20794): Destroying interpreters.

[Wed Feb 11 03:39:29.238348 2015] [wsgi:info] [pid 20794:tid 140288199374592] mod_wsgi (pid=20794): Cleanup interpreter ''.

[Wed Feb 11 03:39:29.238532 2015] [wsgi:info] [pid 20794:tid 140288199374592] mod_wsgi (pid=20794): Terminating Python.

[Wed Feb 11 09:39:29.285717 2015] [core:info] [pid 20787:tid 140288199374592] AH00096: removed PID file /home/simamura/webapps/django_math/express/httpd.pid (pid=20787)

[Wed Feb 11 09:39:29.285738 2015] [mpm_event:notice] [pid 20787:tid 140288199374592] AH00491: caught SIGTERM, shutting down

[Wed Feb 11 03:39:29.734300 2015] [wsgi:info] [pid 20794:tid 140288199374592] mod_wsgi (pid=20794): Python has shutdown.

[Wed Feb 11 03:39:29.734328 2015] [wsgi:info] [pid 20794:tid 140288199374592] mod_wsgi (pid=20794): Exiting process 'localhost:20241'.

[Wed Feb 11 09:46:25.739769 2015] [mpm_event:notice] [pid 18462:tid 140264512485120] AH00489: Apache/2.4.12 (Unix) mod_wsgi/4.4.8 Python/2.7.9 configured -- resuming normal operations

[Wed Feb 11 09:46:25.739732 2015] [wsgi:info] [pid 18464:tid 140264512485120] mod_wsgi (pid=18464): Starting process 'localhost:20241' with threads=10.

[Wed Feb 11 09:46:25.739853 2015] [mpm_event:info] [pid 18462:tid 140264512485120] AH00490: Server built: Feb  6 2015 11:47:32

[Wed Feb 11 09:46:25.739875 2015] [core:notice] [pid 18462:tid 140264512485120] AH00094: Command line: 'httpd (mod_wsgi-express)    -f /home/simamura/webapps/django_math/express/httpd.conf -D WSGI_ACCESS_LOG -D WSGI_MPM_ENABLE_EVENT_MODULE -D WSGI_MPM_EXISTS_EVENT_MODULE -D WSGI_MPM_EXISTS_WORKER_MODULE -D WSGI_MPM_EXISTS_PREFORK_MODULE'

[Wed Feb 11 09:46:25.739940 2015] [wsgi:info] [pid 18464:tid 140264512485120] mod_wsgi (pid=18464): Python home /usr/local.

[Wed Feb 11 09:46:25.739956 2015] [wsgi:info] [pid 18464:tid 140264512485120] mod_wsgi (pid=18464): Initializing Python.

[Wed Feb 11 09:46:25.757249 2015] [wsgi:info] [pid 18464:tid 140264512485120] mod_wsgi (pid=18464): Attach interpreter ''.

[Wed Feb 11 09:46:25.758179 2015] [wsgi:info] [pid 18464:tid 140264512485120] mod_wsgi (pid=18464): Imported 'mod_wsgi'.

[Wed Feb 11 09:46:25.758222 2015] [wsgi:info] [pid 18464:tid 140264512485120] mod_wsgi (pid=18464, process='localhost:20241', application=''): Loading WSGI script '/home/simamura/webapps/django_math/express/handler.wsgi'.

On on the frontend

2015/02/11 09:39:31 [error] 32523#0: *34366513 connect() failed (111: Connection refused) while connecting to upstream

2015/02/11 09:39:41 [error] 32523#0: *34366812 connect() failed (111: Connection refused) while connecting to upstream

Time is same.

What should I do. Maybe change webfaction to something else? Can you advise?

Thank you.

[Graham Dumpleton]

The 502 is issued by upstream nginx and not Apache.

The logs:

[Wed Feb 11 03:39:29.734328 2015] [wsgi:info] [pid 20794:tid 140288199374592] mod_wsgi (pid=20794): Exiting process 'localhost:20241'.
[Wed Feb 11 09:46:25.739769 2015] [mpm_event:notice] [pid 18462:tid 140264512485120] AH00489: Apache/2.4.12 (Unix) mod_wsgi/4.4.8 Python/2.7.9 configured -- resuming normal operations

suggest  that something killed your whole Apache instance and the cron job then restarted it on the quarter hour again.

The log message before what you posted may gibe more information as to why and what signal Apache was sent.

I don't know what WebFaction may do to proactively kill processes.

Graham

[Paul Royik]

What if I make cron to restart Apache every 5 minutes?

Will it crash server? What are consequences?

[Graham Dumpleton]

On 11/02/2015, at 9:36 PM, Paul Royik <distan...@gmail.com> wrote:

> What if I make cron to restart Apache every 5 minutes?

If you look at the existing cron entry, it isn't doing a 'restart' but a 'start'. This means that if Apache is already running it will effectively leave it alone.

Triggering a 'restart' would be bad as you would interrupt running requests every time.

The 'start' ensures that something happens only when it isn't running already.

> Will it crash server? What are consequences?

No, but it is a band aid fix when you should try and work out what may have sent a signal to Apache in the first place to cause it to shutdown. It should not just shutdown itself.

If you look back through the logs, if it was one of the normal expected signals which cause an orderly shutdown, as the last past of the logs before the restart suggest, Apache would have logged what the signal was so you at least know that much.

if you are adamant that you didn't accidentally somehow shut it down, you might want to read up the WebFaction documentation to see if that have a system which identifies memory hungry web server instances and which tries to kill the main parent process. Usually ISPs though they just kill the memory hungry process, which means Apache would have just replaced it and your application would have kept running. The logs suggest that the Apache parent process was shutdown and not just the child process the WSGI application runs in.

Graham

[Paul Royik]

The issue is with memory.

I noticed, that memory is increasing on every Django request.

I found similar question here: http://stackoverflow.com/questions/2293333/django-memory-usage-going-up-with-every-request

Is there any way to avoid it?

And perhaps last question, is I upgrade memory to 1 GB (I have 512 MB), what will be, when traffic increases? I mean for how many visits will it suffice?

If I now have 3000 uniques per day and not enough 512 MB, will this mean that 1 GB is not enough for 6000 uniques? Or relationship is completely another?

Thank you.

[Paul Royik]

Is it also possible to limit memory? Like using resources.setrlimit or something similar? Just raise exception if process takes too long or takes too much memory?

[Graham Dumpleton]

You don't need to resend the email. I know it was there. It has been a very busy week. :-)

On 13/02/2015, at 8:55 PM, Paul Royik <distan...@gmail.com> wrote:

The issue is with memory.

I noticed, that memory is increasing on every Django request.

I found similar question here: http://stackoverflow.com/questions/2293333/django-memory-usage-going-up-with-every-request

Is there any way to avoid it?

Work out why your code is doing.

Is your code runnable on your own computer system? To work this out on Webfaction is going to be too hard. By being able to run it on your own system in a development environment will make it a lot easier for you to isolate the problem as you can request one URL at a time repeatable and what the memory usage for each to see if a particular URL causes the problem.

And perhaps last question, is I upgrade memory to 1 GB (I have 512 MB), what will be, when traffic increases? I mean for how many visits will it suffice?

If I now have 3000 uniques per day and not enough 512 MB, will this mean that 1 GB is not enough for 6000 uniques? Or relationship is completely another?

Impossible for me to tell. You really need proper monitoring for your web application to be able to answer those questions, but you have enough issues now that telling you how to add more stuff, which will use more memory, is not the best idea.

Is it also possible to limit memory? Like using resources.setrlimit or something similar?

When limiting memory you can set a hard limit, but if you reach the limit what your application will do is unpredictable. You aren't usually going to get a nice exception saying you are out of memory, instead it will just crash usually.

Just raise exception if process takes too long or takes too much memory?

Not sure what you mean by 'process' takes too long. For memory, sure you could add a background thread which monitors memory and when it exceeds some level to send a signal to itself to shutdown, causing Apache/mod_wsgi to restart it automatically.

Graham

[Paul Royik]

Thank you for your response.

As I told you, I have integral calculator. In general it is a hard problem.

When algoritm is executed, it goes through a list of integration rules. Sometimes rules can be run infinitely and it is impossible to tell, whether it is infinite execution. Like for integral sin(x)/(cos(x)+tan(x)) algo uses a list of rules and each rule is followed by other rules etc.

So, I set a limit of time, using the following decorator:

def time_limit(timeout):

    def internal(function):

        def internal2(*args, **kw):

            class Calculator(threading.Thread):

                def __init__(self):

                    threading.Thread.__init__(self)

                    self.result = None

                    self.error = None

                def run(self):

                    try:

                        self.result = function(*args, **kw)

                    except Exception as e:

                        self.error = e

                def _stop(self):

                    if self.isAlive():

                        threading.Thread._Thread__stop(self)

            c = Calculator()

            c.start()

            c.join(timeout)

            if c.isAlive():

                raise TimeoutException

            if c.error:

                raise c.error

            return c.result

        return internal2

    return internal

It works good. I usually set limit 150 seconds. But it appears that memory is used faster than time. Even time reduction to 45 seconds doesn't work.

So, I thought that maybe similar code can be written for memory management. When memory is exceeded raise some Exception.

Though, I can't find any working example.

Maybe you can help me?

[Graham Dumpleton]

You aren't getting what I mean.

Have you determined whether your process grows memory usage purely form the Django parts of your application?

In other words, if you ran up the application and hit a URL which doesn't run your algorithm, does it still grow in memory usage?

Make sure it isn't something else in your code.

Also, that you are using a thread to run stuff in immediately looks dodgy.

When you do a join() with a timeout, sure it returns, but it isn't going to kill the thread, it will keep running. You have to set some flag which the algorithm checks to periodically to see if it should exit. I can't see that you are doing that.

Graham

[Paul Royik]

I'm sure, that this algorithm is a root of problem.

Other pages are not a problem.

How can I limit a time wthout a thread or stop it?

For now code is following:

@time_limit(45)

def decorator(my_params):

  return integrate(my_params)

def my_views(request):

    try:

       result = decorator(my_params)

   except TimeOutException: #actually any exception

       # how to stop a thread????

       result = ''

   return HttpResponse(.....)

How to stop a thread?