Launching ERPNext in Production Mode Using supervisord

1,252 views
Skip to first unread message

Christoph H. Larsen

unread,
Jul 19, 2014, 1:04:22 AM7/19/14
to erpnext-dev...@googlegroups.com
Dear All,

I have successfully installed ERPNext in a FreeBSD jali, tricking it into believing that my remote MySQL/MariaSB server (running in a different jail) is actually serving a locally available socket, and managed to use a remote Redis instance.
I have also configured an include on my existing reverse Nginx proxy (running in a different jail) to server dynamic contents via the application server (frappe and friends), and statric contents directly, including the related erpnext/public, frappe/public and shopping_cart/public folder contents using appriate /etc/fstab entries and nullfs mounting.
I DID run the git clone and installation script as root, because I DO want to see application code have root:wheel permissions, run the application as frappe_instance user and loosen up the permissions on specific folders accordingly. Here is the code to do so:
chown -R frappe_caocuero:wheel /usr/local/frappe_instance/logs
chown
-R frappe_instance:www /usr/local/frappe_instance/sites
chmod
-R 640 /usr/local/frappe_instance/logs /usr/local/frappe_instance/sites
chmod
-R 440 /usr/local/frappe_instance/sites/*.txt /usr/local/frappe_instance/sites/assets
chmod -R ug+X /usr/local/frappe_instance/logs /usr/local/frappe_instance/sites
ln -fs /usr/local/frappe_instance/logs /var/log/frappe_instance
chmod 750 /usr/local/frappe_instance/sites
chmod 400 /usr/local/frappe_instance/sites/common_site_config.json /usr/local/frappe_instance/sites/${SITE_ID}/site_config.json
The above configuration works great with:
su - frappe_instance
cd
/usr/local/frappe_instance
./env/bin/honcho start
It works equally well with the stanza recommended after each successful run of the installation script:
su - frappe_instance
cd
/usr/local/frappe_instance
/usr/local/frappe_instance/env/bin/frappe --serve my_site --port 8080 --sites_path /usr/local/frappe_instance/sites
However, the above modes to get ERPNext running are for development, not production mode.
Although I use gunicorn extensively for my existing Django applications (using Dan Bernstein's daemontools), I seem to be out of luck with the configuration file that is provided after successful setup by the ERPNext installation script:
[program:frappe-web]
environment
=SITES_PATH='/usr/local/frappe_instance/sites'
command
=/usr/local/frappe_instance/env/bin/gunicorn -b 127.0.1.113:8080 -w 2 -t 120 frappe.app:application
autostart
=true
autorestart
=true
stopsignal
=QUIT
stdout_logfile
=/usr/local/frappe_instance/logs/web.log
stderr_logfile
=/usr/local/frappe_instance/logs/web.error.log
user
=frappe_instance

[program:frappe-worker]
command
=/usr/local/frappe_instance/env/bin/python -m frappe.celery_app worker
autostart
=true
autorestart
=true
stopsignal
=QUIT
stdout_logfile
=/usr/local/frappe_instance/logs/worker.log
stderr_logfile
=/usr/local/frappe_instance/logs/worker.error.log
user
=frappe_instance
directory
=/usr/local/frappe_instance/sites

[program:frappe-workerbeat]
command
=/usr/local/frappe_instance/env/bin/python -m frappe.celery_app beat -s test.schedule
autostart
=true
autorestart
=true
stopsignal
=QUIT
stdout_logfile
=/usr/local/frappe_instance/logs/workerbeat.log
stderr_logfile
=/usr/local/frappe_instance/logs//workerbeat.error.log
user
=frappe_instance
directory
=/usr/local/frappe_instance/sites

[group:frappe]
programs
=frappe-web,frappe-worker,frappe-workerbeat
Strangely enough, the only error I get is a generic "Internal Server Error", without any /usr/local/frappe_instance/logs/ entries.
Do you have any hints on where to look, and what to do?
Thanks a lot, indeed!

Chris

Pratik Vyas

unread,
Jul 19, 2014, 4:12:04 AM7/19/14
to erpnext-dev...@googlegroups.com
On Sat, Jul 19, 2014 at 10:34 AM, Christoph H. Larsen
<christoph...@gmail.com> wrote:
> Dear All,
>
> I have successfully installed ERPNext in a FreeBSD jali, tricking it into
> believing that my remote MySQL/MariaSB server (running in a different jail)
> is actually serving a locally available socket, and managed to use a remote
> Redis instance.
> I have also configured an include on my existing reverse Nginx proxy
> (running in a different jail) to server dynamic contents via the application
> server (frappe and friends), and statric contents directly, including the
> related erpnext/public, frappe/public and shopping_cart/public folder
> contents using appriate /etc/fstab entries and nullfs mounting.

Hi Chris,

This is great! However, I am inexperienced with BSD jails ( the only
BSD I ever used was Mac OS X ;) )

> I DID run the git clone and installation script as root, because I DO want
> to see application code have root:wheel permissions, run the application as
> frappe_instance user and loosen up the permissions on specific folders
> accordingly. Here is the code to do so:

Is this because it would disallow anyone to change the application
code if a site is compromised?

For public files, we have a collectstatic like system in Django, so
all you need to serve is sites/public and ${SITE_ID}/public. The publc
files from the apps (frappe/erpnext) are symlinked to this directory.
Just in case you face permission errors after you get gunicorn
working.
Can you make sure if the supervisor was able to get gunicorn up?
`supervisorctl status`. Even if gunicorn was not up, nginx should
throw "Bad Gateway"?

Can you check if the 500 was from nginx ( and the request didn't even
reach gunicorn)?

If the "Internal Server Error" is from gunicorn, then there has to be
an entry in bench/logs/web.error.log (if it's able to open this file,
I see that you've set permissions for this).

To debug, you can also stop supervisor and run the gunicorn command
directly as frappe_instance and see the errors on stderr.

If you were able to successfully reverse proxy the development server,
the only difference I can think of is that the development server
binds to 0.0.0.0 and with the production settings, gunicorn is
configured to bind to a loopback address.


Thanks,
--
Pratik
erpnext

Christoph H. Larsen

unread,
Jul 20, 2014, 10:52:00 AM7/20/14
to erpnext-dev...@googlegroups.com
Dear Pratik,
Thanks a lot for your reply. Please see my comments inline for easier follow-up:


On Saturday, 19 July 2014 15:12:04 UTC+7, Pratik Vyas wrote:
On Sat, Jul 19, 2014 at 10:34 AM, Christoph H. Larsen
<christoph...@gmail.com> wrote:
> Dear All,
>
> I have successfully installed ERPNext in a FreeBSD jali, tricking it into
> believing that my remote MySQL/MariaSB server (running in a different jail)
> is actually serving a locally available socket, and managed to use a remote
> Redis instance.
> I have also configured an include on my existing reverse Nginx proxy
> (running in a different jail) to server dynamic contents via the application
> server (frappe and friends), and statric contents directly, including the
> related erpnext/public, frappe/public and shopping_cart/public folder
> contents using appriate /etc/fstab entries and nullfs mounting.

Hi Chris,

This is great! However, I am inexperienced with BSD jails ( the only
BSD I ever used was Mac OS X ;) )
I guess it is worth our while to chew our way through it, because it would yield installation instructions for a truly scalable architecture... Thanks for bearing with me.

> I DID run the git clone and installation script as root, because I DO want
> to see application code have root:wheel permissions, run the application as
> frappe_instance user and loosen up the permissions on specific folders
> accordingly. Here is the code to do so:

Is this because it would disallow anyone to change the application
code if a site is compromised?
Exactly. Luckily, the permission structure is pretty easy with Frappe/ERPNext, as provided below using the short chown/chmod stanza sent yesterday.

For public files, we have a collectstatic like system in Django, so
all you need to serve is sites/public and ${SITE_ID}/public. The publc
files from the apps (frappe/erpnext) are symlinked to this directory.
Just in case you face permission errors after you get gunicorn
working.
I got stuck here (and yes, I know Django's collectstatic system quite well): There IS a ${INSTANCE}/sites/${SITE_ID}/public directory, but I fail to see a ${INSTANCE}/sites/public folder. It would of course be great, is some collectstatic facility sticks all the apps' assets in there. Symlinks are not always doable with Nginx, which tends to ignore them, even with the right configuration settings. Guess, it's a feature, not a bug. I feel that getting the apps' public assets in is the most important issue, but as long as there is no ${INSTANCE}/sites/public folder I may be stuck.
I assume that we can get on top of gunicorn and such, as soon as the public assets folder issue for te apps has been sorted out; hence I am holding my horses in the issues below.
Can you make sure if the supervisor was able to get gunicorn up?
`supervisorctl status`. Even if gunicorn was not up, nginx should
throw "Bad Gateway"?

Can you check if the 500 was from nginx ( and the request didn't even
reach gunicorn)?

If the "Internal Server Error" is from gunicorn, then there has to be
an entry in bench/logs/web.error.log (if it's able to open this file,
I see that you've set permissions for this).

To debug, you can also stop supervisor and run the gunicorn command
directly as frappe_instance and see the errors on stderr.

If you were able to successfully reverse proxy the development server,
the only difference I can think of is that the development server
binds to 0.0.0.0 and with the production settings, gunicorn is
configured to bind to a loopback address.
Sure enough, I gave it the local jail's I.P. and an appropriate port number.


Thanks,
Thanks a million!
--
Pratik
Chris
 
erpnext

Pratik Vyas

unread,
Jul 20, 2014, 12:09:14 PM7/20/14
to erpnext-dev...@googlegroups.com
> I guess it is worth our while to chew our way through it, because it would
> yield installation instructions for a truly scalable architecture... Thanks
> for bearing with me.

Yes! We should do this.


> I got stuck here (and yes, I know Django's collectstatic system quite well):
> There IS a ${INSTANCE}/sites/${SITE_ID}/public directory, but I fail to see
> a ${INSTANCE}/sites/public folder. It would of course be great, is some
> collectstatic facility sticks all the apps' assets in there. Symlinks are
> not always doable with Nginx, which tends to ignore them, even with the
> right configuration settings. Guess, it's a feature, not a bug. I feel that
> getting the apps' public assets in is the most important issue, but as long
> as there is no ${INSTANCE}/sites/public folder I may be stuck.

My answer was incomplete. The ${INSTANCE}/sites/assets directory is
what you're looking for. The structure is as follows.


sites/assets -> For the static files across the apps. This is
generated using the `frappe -b` command. Symlinked by default and
copies complete files if called with `--make_copy` option. One
limitation with make_copy at the moment is that for a rebuild, you'll
have to remove the existing assets directory and run the command
because it doesn't replace existing files. Maps to STATIC_DIR in
Django.

sites/${SITE}/public -> For storing attachments. Maps to MEDIA_DIR in Django.


> I assume that we can get on top of gunicorn and such, as soon as the public
> assets folder issue for te apps has been sorted out; hence I am holding my
> horses in the issues below.

Yes. If supervisor is the culprit, configuring daemontools shouldn't
be much of an effort.

Thanks for your interest in Frappe :)
--
Pratik
erpnext

Christoph H. Larsen

unread,
Jul 21, 2014, 9:27:38 AM7/21/14
to erpnext-dev...@googlegroups.com
Dear Pratik,

The following is a little update on my installation adventure on FreeBSD across different jails for mail, reverse proxy, database and application:


On Sunday, 20 July 2014 23:09:14 UTC+7, Pratik Vyas wrote:
> I guess it is worth our while to chew our way through it, because it would
> yield installation instructions for a truly scalable architecture... Thanks
> for bearing with me.

Yes! We should do this.
Agreed. Last not least, your efforts deserve this :-).


> I got stuck here (and yes, I know Django's collectstatic system quite well):
> There IS a ${INSTANCE}/sites/${SITE_ID}/public directory, but I fail to see
> a ${INSTANCE}/sites/public folder. It would of course be great, is some
> collectstatic facility sticks all the apps' assets in there. Symlinks are
> not always doable with Nginx, which tends to ignore them, even with the
> right configuration settings. Guess, it's a feature, not a bug. I feel that
> getting the apps' public assets in is the most important issue, but as long
> as there is no ${INSTANCE}/sites/public folder I may be stuck.

My answer was incomplete. The ${INSTANCE}/sites/assets directory is
what you're looking for. The structure is as follows.


sites/assets -> For the static files across the apps. This is
generated using the `frappe -b` command. Symlinked by default and
copies complete files if called with `--make_copy` option. One
limitation with make_copy at the moment is that for a rebuild, you'll
have to remove the existing assets directory and run the command
because it doesn't replace existing files. Maps to STATIC_DIR in
Django.

sites/${SITE}/public -> For storing attachments. Maps to MEDIA_DIR in Django.
Understood. I did the --make_copy trick, and it avoided placing a symlink into a folder that was to be union mounted  - because this is not permissible. I do note that I will have to delete the static asset directories prior to any upgrading. This has greatly simplified the mounts required for my reverse proxy and application jails. I essentially make a big enough data partition available for ERPNext that I mount on ${INSTANCE}/sites/${SITE_ID}/public as file repository. Likewise, I mount the application server jail's $INSTANCE}/sites/${SITE_ID}/public on the /home/${INSTANCE}/sites/${SITE_ID}/public  of my reverse proxy jail, for access by Nginx. The same applies to the application server jail's ${INSTANCE/sites/assets that is mounted on the reverse proxy jail's /home/${INSTANCE}/sites/assets folder. So far, so good.
honcho starts, run as user ${INSTANCE}, without problems.
BUT: The Nginx nginx.conf file that comes with ERPNext does not float: I had to comment out the /assets stanza, as these very assets arre simply not servable via the reverse proxy. WEIRD. Here is the cuplrit (relevant snippet, only), with INSTANCE=frappe_erpnext and SITE_ID=erpnext
# Define the document root of your site
root                            
/usr/local/frappe_erpnext/sites;

# Define the location of your site's private files
location                        
/private/ {
   
internal;
    try_files                  
/$uri =424;
}

# Define the location of your sites assets
#location                       /assets {
#    try_files                  $uri =404;
#}


# handle default location
location                        
/ {
   
# naxsi rules, apply to / as first configuration entry
    include                     naxsi_default
.rules;
    try_files                  
/erpnext/public/$uri @magic;
}

# handle proxy location
location
@magic {
    proxy_set_header X
-Use-X-Accel-Redirect True;
    proxy_pass                  http
://www_py.jail.vlan:8080;
}
(The above settings work on the https side of thing,s only, as all traffic is redirected there. Likewise, it is part of a bigger /include framework.)
If I uncomment the settings of the "/assets" stanza, I get raw .html, only, no js, no css, as it is seemingly not accessible via Nginx at the prescribed location. The rest works.
Of course, we do want static assets served by Nginx for sheer simplicity and raw speed - so this should be fixed.
Next I tried to tackle gunicorn by its horns - virtually. I use it for a lot of Python applications, but seem to get lost here. Avoiding supervisord, I just issued, from within the application server jail the following from three different consoles:
/usr/local/${INSTANCE}/env/bin/gunicorn -b 127.0.1.113:8080 -w 2 -t 120 -u ${INSTANCE} --error-logfile /var/log/${INSTANCE}/gunicorn.log frappe.app:application
su $
{INSTANCE} -c "/usr/local/${INSTANCE}/env/bin/python -m frappe.celery_app worker"
su $
{INSTANCE} -c "/usr/local/${INSTANCE}/env/bin/python -m frappe.celery_app beat -s test.schedule"
While the two celery application start up without flaw and produce proper log entries. Likewise, the gunicorn error log tells me that all is fine, and that gunicorn has started up smoothly.
This is, however, not translated into a functioning ERPNext: It get an "Internal Server Error" that is not logged by gunicorn, nor by the application: ${INSTANCE}/logs/web*.log remain empty, unlike with honcho (which works).
WEIRD.
Yes. If supervisor is the culprit, configuring daemontools shouldn't
be much of an effort.
I don't think that supervisord is the culprit, but will use daemontools as I use them inside this jail already anyway, to control other Django instances.

Thanks for your interest in Frappe :)
Thank YOU for your ongoing help and support! Hugely appreciated.
--
Pratik
erpnext
 Chris

Christoph H. Larsen

unread,
Jul 23, 2014, 11:17:38 PM7/23/14
to erpnext-dev...@googlegroups.com

Dear Pratik,
Gnawing my way through...
First, the gunicorn commands as issued via the supervisord script (or manually for that matter) are definitely correct.
su - ${INSTANCE}
setenv SITES_PATH
/usr/local/${INSTANCE}/sites    (for *BSD, or:)
SITES_PATH
='/usr/local/${INSTANCE}/sites'    (for *nix)
/usr/local/$INSTANCE}/env/bin/gunicorn -b ${IP}:${PORT} -w 2 -t 120 app:application
definitely works and gives a nice log:
2014-07-24 05:01:56 [77304] [INFO] Worker exiting (pid: 77304)
2014-07-24 05:02:13 [77364] [INFO] Starting gunicorn 19.0.0
2014-07-24 05:02:13 [77364] [INFO] Listening at: http://127.0.1.113:8080 (77364)
2014-07-24 05:02:13 [77364] [INFO] Using worker: sync
2014-07-24 05:02:13 [77366] [INFO] Booting worker with pid: 77366
2014-07-24 05:02:13 [77367] [INFO] Booting worker with pid: 77367
However, accessing this site either locally or remotely (via Nginx reverse proxy) yields a persistent 404 error.
This IS strange. Any thoughts?
Only once we got this sorted, I will get going with the Nginx configuration, which does not seem to work even under honcho, as described before.
Thanks a lot for your help!
Chris
 

Pratik Vyas

unread,
Jul 24, 2014, 1:08:42 AM7/24/14
to erpnext-dev...@googlegroups.com
This could be easy to solve. Your Host header has to match the
sitename for Frappe to select the site. To make a site be served as
default what I normally do is hard code the header in the proxy config
of nginx. In most production settings, the dnsname and hostname are
same, so it works dynamically that way.

> This IS strange. Any thoughts?
> Only once we got this sorted, I will get going with the Nginx configuration,
> which does not seem to work even under honcho, as described before.

The nginx issue does seem strange to me.
> Thanks a lot for your help!

Thanks :)
Also, I am curious, is your test machine a local vm? Are there any
cheap (per hour basis, like on digital ocean) FreeBSD vpses available?

> Chris
>
>
> --
> Note:
>
> If you are posting an issue,
>
> 1. ERPNext is a free and open source software and support is given on this
> forum by a team (https://frappe.io/webnotes). So please consider donating if
> you find this forum useful (https://frappe.io/buy). Even a small amount
> would be helpful.
> 2. We should be able to replicate it at our end. So please give us as much
> information as you can. Please see it from the point of view of the person
> receiving the communication.
> 3. Paste your code at http://pastebin.com or http://gist.github.com and send
> only the URL via email
> 4. For sending images, use http://imgur.com or other similar services. Do
> not send images as attachments. Links are good. Same goes for any file you
> are going to send.
>
> End of Note
> ---
> You received this message because you are subscribed to the Google Groups
> "ERPNext Developer Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to erpnext-developer...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/erpnext-developer-forum/8faadd2a-6796-490e-bb57-13dd4dc650c1%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Pratik
erpnext

Christoph H. Larsen

unread,
Jul 24, 2014, 5:05:33 AM7/24/14
to erpnext-dev...@googlegroups.com
Dear Pratik,
Thanks a lot for your reply. Please see my inline comments below:
The issue described today has nothing to do with Nginx - I have deliberately tested just ERPNext with gunicorn (with the command parameters prescirbed in the supervisord.conf file that comes with ERPNext), and left Nginx completely out of the picture: I simply try to access the running gunicorn instance from a different jail, firewall settings permitting.
> This IS strange. Any thoughts?
> Only once we got this sorted, I will get going with the Nginx configuration,
> which does not seem to work even under honcho, as described before.

The nginx issue does seem strange to me.
> Thanks a lot for your help!

Thanks :)
Also, I am curious, is your test machine a local vm? Are there any
cheap (per hour basis, like on digital ocean) FreeBSD vpses available?
I have a dedicated root server with Hetzner (www.hetzner.de), run FreeBSD on the system throughout, and FreeBSD jails for each service, as required. AFAIK RootBSD does VMs. However, with the Hetzner pricing, it's just not worth while.
Thanks a lot for your help!

> Chris
>
>
> --

Pratik Vyas

unread,
Jul 24, 2014, 5:14:37 AM7/24/14
to erpnext-dev...@googlegroups.com
> The issue described today has nothing to do with Nginx - I have deliberately
> tested just ERPNext with gunicorn (with the command parameters prescirbed in
> the supervisord.conf file that comes with ERPNext), and left Nginx
> completely out of the picture: I simply try to access the running gunicorn
> instance from a different jail, firewall settings permitting.
>>

Yes, it's responding with 404 because when running with gunicorn,
Frappe has to figure out what site it has to serve (from the sites
directory). That is decided based on the Host header of the HTTP
request. When you run hocho start, it automatically serves the site
name in sites/currentsite.txt

So, to test outside the jail, try running the command as

curl -I http://jail_ip:8000 -H "Host: $SITE_NAME"


>> > This IS strange. Any thoughts?
>> > Only once we got this sorted, I will get going with the Nginx
>> > configuration,
>> > which does not seem to work even under honcho, as described before.
>>
>> The nginx issue does seem strange to me.
>> > Thanks a lot for your help!
>>
>> Thanks :)
>> Also, I am curious, is your test machine a local vm? Are there any
>> cheap (per hour basis, like on digital ocean) FreeBSD vpses available?
>
> I have a dedicated root server with Hetzner (www.hetzner.de), run FreeBSD on
> the system throughout, and FreeBSD jails for each service, as required.
> AFAIK RootBSD does VMs. However, with the Hetzner pricing, it's just not
> worth while.
> Thanks a lot for your help!

Yes, FreeBSD hosting services seem fewer. I guess, am better off
running on a local virtualbox to try this.

>>
>>
>> > Chris


Thanks!
--
Pratik
erpnext

Christoph H. Larsen

unread,
Jul 24, 2014, 11:09:30 AM7/24/14
to erpnext-dev...@googlegroups.com
Dear Pratik,
My feedback below, as usual:


On Thursday, 24 July 2014 16:14:37 UTC+7, Pratik Vyas wrote:
> The issue described today has nothing to do with Nginx - I have deliberately
> tested just ERPNext with gunicorn (with the command parameters prescirbed in
> the supervisord.conf file that comes with ERPNext), and left Nginx
> completely out of the picture: I simply try to access the running gunicorn
> instance from a different jail, firewall settings permitting.
>>

Yes, it's responding with 404 because when running with gunicorn,
Frappe has to figure out what site it has to serve (from the sites
directory). That is decided based on the Host header of the HTTP
request. When you run hocho start, it automatically serves the site
name in sites/currentsite.txt
Yep, this sounds trivial - but it isn' - quitet. My setup (which works flawlessly with WebDAV, Ruby on Rails, Django, Pyramid, even PHP ;-)!)  is to blame:
FreeBSD jails for Nginx as reverse proxy, and for ERPNext (with gunicorn, which start alright, but yields 404 errors, as you know).
The $host header (which I never hardcode) is certainly not my ${SITE_ID} (erpnext). I can hard code the $host variable, but to what? The jail name of the ERPNext server instance? As http://[IP number] or just as [IP number]? Or the FQDN?
Things don't seem to work - yet.

Thanks a lot for your help!
Chris

Christoph H. Larsen

unread,
Jul 24, 2014, 9:39:47 PM7/24/14
to erpnext-dev...@googlegroups.com
Dear Pratik,
A little update:


On Thursday, 24 July 2014 16:14:37 UTC+7, Pratik Vyas wrote:
> The issue described today has nothing to do with Nginx - I have deliberately
> tested just ERPNext with gunicorn (with the command parameters prescirbed in
> the supervisord.conf file that comes with ERPNext), and left Nginx
> completely out of the picture: I simply try to access the running gunicorn
> instance from a different jail, firewall settings permitting.
>>

Yes, it's responding with 404 because when running with gunicorn,
Frappe has to figure out what site it has to serve (from the sites
directory). That is decided based on the Host header of the HTTP
request. When you run hocho start, it automatically serves the site
name in sites/currentsite.txt

So, to test outside the jail, try running the command as

curl -I http://jail_ip:8000 -H "Host: $SITE_NAME"
First, I disabled all inter-jail firewalling.
Second, I made sure, the application server is accessible from the jail I am testing from:
telnet [ERPNext jail IP] [ERPNext port]
(works)
Third, I issued:
curl -I http://[ERPNext jail IP]:[ERPNext port] -H "Host: $SITE_NAME"
This gives me:
"SITE_NAME: Undefined variable"
Fourth, I tried:
curl -I http://[ERPNext jail IP]:[ERPNext port] -H "Host: erpnext"
("erpnext" is my site ID)
This provides:
HTTP/1.0 404 NOT FOUND
Content-Type: text/html; charset: utf-8
Content-Length: 7456
X
-Page-Name: 404
X
-From-Cache: False
Set-Cookie: sid=Guest; Expires=Mon, 28-Jul-2014 03:33:55 GMT; Path=/
Server: Werkzeug/
0.9.6 Python/2.7.8
Date: Fri, 25 Jul 2014 01:33:55 GMT
(Still: a 404 error)
Stumped... Thank you so much for your ideas!

Pratik Vyas

unread,
Jul 25, 2014, 9:44:30 AM7/25/14
to erpnext-dev...@googlegroups.com
On Thu, Jul 24, 2014 at 8:39 PM, Christoph H. Larsen
<christoph...@gmail.com> wrote:
> Dear Pratik,
> My feedback below, as usual:
>
>
> On Thursday, 24 July 2014 16:14:37 UTC+7, Pratik Vyas wrote:
>>
>> > The issue described today has nothing to do with Nginx - I have
>> > deliberately
>> > tested just ERPNext with gunicorn (with the command parameters
>> > prescirbed in
>> > the supervisord.conf file that comes with ERPNext), and left Nginx
>> > completely out of the picture: I simply try to access the running
>> > gunicorn
>> > instance from a different jail, firewall settings permitting.
>> >>
>>
>> Yes, it's responding with 404 because when running with gunicorn,
>> Frappe has to figure out what site it has to serve (from the sites
>> directory). That is decided based on the Host header of the HTTP
>> request. When you run hocho start, it automatically serves the site
>> name in sites/currentsite.txt
>
> Yep, this sounds trivial - but it isn' - quitet. My setup (which works
> flawlessly with WebDAV, Ruby on Rails, Django, Pyramid, even PHP ;-)!) is
> to blame:

I think it is just a configuration issue. We'll get through it.

> FreeBSD jails for Nginx as reverse proxy, and for ERPNext (with gunicorn,
> which start alright, but yields 404 errors, as you know).
> The $host header (which I never hardcode) is certainly not my ${SITE_ID}
> (erpnext). I can hard code the $host variable, but to what? The jail name of
> the ERPNext server instance? As http://[IP number] or just as [IP number]?
> Or the FQDN?
> Things don't seem to work - yet.
>

You have to hardcode the $host variable to the site name ie.
$INSTANCE/sites/site_name (which in your case). If you don't like
hardcoding the HOST header, you can set X-Frappe-Site-Name header to
the site_name.

Thanks,
--
Pratik
erpnext

Christoph H. Larsen

unread,
Jul 26, 2014, 4:22:07 AM7/26/14
to erpnext-dev...@googlegroups.com
Dear Pratik,
As always, a huge thank you for your help. My progress below:
I have set the following in Nginx:
server {
   
[...]
   
proxy_set_header                X-Frappe-Site-Name /usr/local/frappe_erpnext/sites/erpnext


   
# Define the document root of your site
    root                            
/usr/local/frappe_erpnext/sites;

   
# Define the location of your site's private files
    location                        
/private/ {
         
internal;
         try_files                  
/$uri =424;
   
}

   
# Define the location of your sites assets

    location                        
/assets {
         try_files                   $uri
=404;
   
}

    location                        
/ {

        include                     naxsi_default
.rules;
        try_files                  
/erpnext/public/$uri @magic;
   
}

   
# handle proxy location
    location
@magic {

        proxy_pass                  http
://[ERPNext application server jail]:8080;
   
}
}
Partial success ith gunicorn: no more 404 error. However, with gunicorn, the /assets in the above /assets stanza were still not seen, so all I got was a raw website. I get the same, if I comment out the /assets stanza.
With honcho I cannot see any /assets (= raw website), unless I comment out the /assets stanza above.
Baby steps, after all. How can we sort out the inaccessible /assets issue? double-checked permissions - looks all fine, and gunicorn is running under the same user as honcho, Nginx can read union mounted assets folders.

Thanks a lot for your help!
Chris

Thanks,
--
Pratik
erpnext

Pratik Vyas

unread,
Jul 26, 2014, 1:09:59 PM7/26/14
to erpnext-dev...@googlegroups.com
Hi Chris,

Great! It's weird that you have to put the whole absolute path in the header.
>
> location /assets {
> try_files $uri =404;
> }
>

Just noticed this in your config. Can you try changing $uri to /$uri ?

Thanks!
--
Pratik
erpnext

Christoph H. Larsen

unread,
Jul 27, 2014, 11:08:53 AM7/27/14
to erpnext-dev...@googlegroups.com
Dear Pratik,


On Sunday, 27 July 2014 00:09:59 UTC+7, Pratik Vyas wrote:
Hi Chris,

Great! It's weird that you have to put the whole absolute path in the header.
>
>     location                        /assets {
>          try_files                   $uri =404;
>     }
>

Just noticed this in your config. Can you try changing $uri to /$uri ?
I tried your suggestion, but no joy :-(.
I managed to change:
proxy_set_header                X-Frappe-Site-Name /usr/local/frappe_erpnext/sites/erpnext;
to:
proxy_set_header                X-Frappe-Site-Name erpnext;
Not sure, why this did not work yesterday.
However, we are still left with raw html when trying gunicorn... Bummer!
Thank you so much in advance!
Chris

Thanks!
--
Pratik
erpnext

Pratik Vyas

unread,
Jul 27, 2014, 11:30:19 AM7/27/14
to erpnext-dev...@googlegroups.com
On Sun, Jul 27, 2014 at 8:38 PM, Christoph H. Larsen
<christoph...@gmail.com> wrote:
> Dear Pratik,
>
>
> On Sunday, 27 July 2014 00:09:59 UTC+7, Pratik Vyas wrote:
>>
>> Hi Chris,
>>
>> Great! It's weird that you have to put the whole absolute path in the
>> header.
>> >
>> > location /assets {
>> > try_files $uri =404;
>> > }
>> >
>>
>> Just noticed this in your config. Can you try changing $uri to /$uri ?
>
> I tried your suggestion, but no joy :-(.
> I managed to change:
> proxy_set_header X-Frappe-Site-Name
> /usr/local/frappe_erpnext/sites/erpnext;
> to:
> proxy_set_header X-Frappe-Site-Name erpnext;
> Not sure, why this did not work yesterday.

Are you sure that you reloaded/restarted nginx in the correct jail ? :)

because /$uri should have worked as it might be looking for assetsabcd
if you requested assets/abcd.

Anything in nginx logs? Maybe if you enable nginx debug log for that
location it might give you a clue.

Thanks,
--
Pratik
erpnext

Christoph H. Larsen

unread,
Jul 29, 2014, 12:15:38 AM7/29/14
to erpnext-dev...@googlegroups.com
Dear Pratik,
Solved! I was forced to make some modifications to the Nginx configuration, and have included them herewith, assuming that the instance is "frappe_erpnext" and the site_id is "erpnext":
# enable proxy cache
proxy_cache                     erp
.caocuero.com_proxy_cache;
expires                        
7d;

# set proxy headers
proxy_set_header                X
-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header                X
-Use-X-Accel-Redirect True;
proxy_set_header                
Host $host;
proxy_set_header                X-Frappe-Site-Name /usr/local/frappe_erpnext/sites/erpnext;
proxy_read_timeout              
120;
proxy_redirect                  off
;


# Define the location of your site's private files

location                        
/private {
   
alias                       /home/frappe_erpnext/sites/private;
   
internal;
    try_files                  
/$uri =424;

}

# Define the location of your application assets
location                        
/assets {
   
alias                       /home/frappe_erpnext/sites/assets;
}

# Define the location of your media files
location                        
/files {
   
alias                       /home/frappe_erpnext/sites/erpnext/public/files;
}

# handle default location
location                        / {
    alias                       /home/frappe_erpnext/sites;
    try_files                   $uri @frappe_erpnext;
}

location                        @frappe_erpnext {
    proxy_pass                  http://[erpnext_application_server]:[erpnext_application_port];
}
Essentially, I had to switch from the "root" concept to the "alias" one, do await with the "try_files" approach in the /assets directory, and specify the media files directory /files.
The only caveat is whether or not ERPNext may create any extra folders for internet access in "${INSTANCE}/sites/erpnext/public", other than "files", in which case I would have to add an extra stanza. Thoughts? (And: Why were these mods required - I am, admittedly, not entirely clear about that!)
With the above modifications for Nginx, the suggested supervisord.conf works like a charm.
Thank you so much for your help and patience!
Chris

Pratik Vyas

unread,
Jul 30, 2014, 6:30:24 AM7/30/14
to erpnext-dev...@googlegroups.com
On Tue, Jul 29, 2014 at 9:45 AM, Christoph H. Larsen
<christoph...@gmail.com> wrote:
> Dear Pratik,
> in which case I would have to add an extra stanza. Thoughts? (And: Why were
> these mods required - I am, admittedly, not entirely clear about that!)
> With the above modifications for Nginx, the suggested supervisord.conf works
> like a charm.
> Thank you so much for your help and patience!

Hey Chris,

Awesome!!

Glad the it now works for you in production. It's strange that
try_files didn't workout but yeah, with alias and splitting assets/
and files/ it should still work fine :)


> The only caveat is whether or not ERPNext may create any extra folders for
> internet access in "${INSTANCE}/sites/erpnext/public", other than "files",

You're covered for that atleast in version 4 as we do not create extra
"public/files" like folders. There used to be "public/backups" but now
it's a private URL.

With version 4 we brought out this seperation in the URL for assets
and now I think that this decision has rewarded us.

Thanks a lot!
--
Pratik
erpnext

Christoph H. Larsen

unread,
Aug 1, 2014, 10:34:12 AM8/1/14
to erpnext-dev...@googlegroups.com

Dear Pratik,
The huge thanks should all be aimed at you and your team!  Let me know, how I can help - am thinking still of the "translatability" of user data entries, as I have seen may deployments of FLOSS ERP (well, Tryton is the only one that cuts it in the truly FLOSS sense!) in the said multi-lingual environments.
Cheers and more soon!
Chris

Rushabh Mehta

unread,
Aug 4, 2014, 1:06:08 AM8/4/14
to erpnext-dev...@googlegroups.com
Dear Pratik,
The huge thanks should all be aimed at you and your team!  Let me know, how I can help


Thanks for the offer Chris. Please help us to spread the word! We are not a "funded" project like some of the other "open source" ERPs out there and depend on the goodwill of our users and community.


- am thinking still of the "translatability" of user data entries, as I have seen may deployments of FLOSS ERP (well, Tryton is the only one that cuts it in the truly FLOSS sense!) in the said multi-lingual environments.
Cheers and more soon!
Chris

--
Note:
 
If you are posting an issue,
 
1. ERPNext is a free and open source software and support is given on this forum by a team (https://frappe.io/webnotes). So please consider donating if you find this forum useful (https://frappe.io/buy). Even a small amount would be helpful.
2. We should be able to replicate it at our end. So please give us as much information as you can. Please see it from the point of view of the person receiving the communication.
3. Paste your code at http://pastebin.com or http://gist.github.com and send only the URL via email
4. For sending images, use http://imgur.com or other similar services. Do not send images as attachments. Links are good. Same goes for any file you are going to send.
 
End of Note
---
You received this message because you are subscribed to the Google Groups "ERPNext Developer Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to erpnext-developer...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages