[wsgi:error] [pid xxxx] Connecting to server...

[Tim Moody]

I am seeing a large number of these in the apache2 error log. The application is an html/javascript front end that makes jquery ajax calls to a mod_wsgi python app that is simply a wrapper to send requests over zeromq to a local custom server written in python. On startup it makes 8 such calls and this error can appear after the 4th or 5th; however all 8 frequently, but not always, succeed. The wrapper talks to the server over an ipc socket.

I'm not clear if this message is coming from mod_wsgi or zeromq or apache or something else. The server being contacted does not show any errors.

[Graham Dumpleton]

> On 9 Aug 2018, at 11:24 pm, Tim Moody <t...@timmoody.com> wrote:
>
> I am seeing a large number of these in the apache2 error log. The application is an html/javascript front end that makes jquery ajax calls to a mod_wsgi python app that is simply a wrapper to send requests over zeromq to a local custom server written in python. On startup it makes 8 such calls and this error can appear after the 4th or 5th; however all 8 frequently, but not always, succeed. The wrapper talks to the server over an ipc socket.
>
> I'm not clear if this message is coming from mod_wsgi or zeromq or apache or something else. The server being contacted does not show any errors.

It looks like logging from your Python application code or some Python package it uses.

Anything logged/printed by the Python web application being hosted, to stdout/stderr, gets saved into the Apache error log. It will be marked as wsgi:error, but is just ordinary program output and need not mean there is an error, and definitely is not from mod_wsgi itself.

Have you tried running 'grep' on your application source code, and your Python environments site-packages directory, to search for the string 'Connecting to server'?

Graham

[Tim Moody]

Thanks for replying so quickly. The message is indeed a print statement in my python wrapper. It was the wsgi.error that threw me off. So I guess it is really an info message.

However, I still get failed connections, but perhaps these are from zeromq and not from mod_wsgi. However, to be sure, I should say that I am not using WSGIDaemonProcess but only WSGIScriptAlias and all other defaults (on 4.3.0 Ubuntu 16.04). Is there any chance that mod_wsgi will be starved for connections if 8 requests arrive nearly simultaneously?

Thanks for your help.

[Graham Dumpleton]

> On 10 Aug 2018, at 9:22 am, Tim Moody <t...@timmoody.com> wrote:
>
> Thanks for replying so quickly. The message is indeed a print statement in my python wrapper. It was the wsgi.error that threw me off. So I guess it is really an info message.
>
> However, I still get failed connections, but perhaps these are from zeromq and not from mod_wsgi. However, to be sure, I should say that I am not using WSGIDaemonProcess but only WSGIScriptAlias and all other defaults (on 4.3.0 Ubuntu 16.04). Is there any chance that mod_wsgi will be starved for connections if 8 requests arrive nearly simultaneously?

Is your use of the client for backend service thread safe?

A typical configuration for Apache/mod_wsgi means that you have multiple threads per process handling requests, so you have to be careful to ensure that each request has own client instance, or if one client instance that can handle multithread access okay.

Anyway, how many concurrent requests mod_wsgi can handle for embedded mode is dictated by Apache MPM settings. This is unrelated though to backend connections and whether limits enforced by backend service. If capacity to handle more requests had been reached as Apache MPM settings low, requests would just queue up and never get to the application until free capacity, so not going to affect anything in backend connections for those requests which are able to be handled.

Graham

[Tim Moody]

Well, I think it is thread safe. The mod_wsgi application is an end point for an ajax request, which it forwards to the multi-threaded backend server over a zeromq socket (using a linux ipc socket), polls for a response for up to 30 seconds, and returns the json result or failure to the ajax caller's callback function. It then terminates. All of this is password protected using apache mod_auth, part of the reason I didn't just use uwsgi.

--
You received this message because you are subscribed to a topic in the Google Groups "modwsgi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/modwsgi/HMKpmmNacX4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to modwsgi+unsubscribe@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Tim Moody]

If it is not an imposition here is the entire app:

import zmq

import os.path

import time

import urllib2

cmdsrv_pid_file = "{{ cmdsrv_pid_file }}"

cmdsrv_ready_file = "{{ cmdsrv_ready_file }}"

alert_param = ',"Alert": "True"}'

def application(environ, start_response):

    start_ms = time.time()

    if environ['REQUEST_METHOD'] == 'POST':

        try:

            request_body_size = int(environ.get('CONTENT_LENGTH', 0))

        except (ValueError):

            request_body_size = 0

        request_body = environ['wsgi.input'].read(request_body_size)

        # request_body holds command=<text of cmd>

        if os.path.exists(cmdsrv_pid_file):

            if os.path.exists(cmdsrv_ready_file):

                # unpack cmd, urldecode, and substitute space for %20

                cmd = urllib2.unquote(request_body.split('=')[1]).decode('utf8')

                cmd = cmd.replace('%20', ' ')

                response_body = send_command(cmd, start_ms)

            else:

                response_body = '{"Error": "CMDSRV has started but is not ready."' + alert_param

        else:

            response_body = '{"Error": "CMDSRV is not running."' + alert_param

        response_headers = [('Content-type', 'application/json'), ('Content-Length', str(len(response_body)))]

    else: # not a POST

        response_body = dump(environ)

        response_headers = [('Content-type', 'text/plain'),

                       ('Content-Length', str(len(response_body)))]

    status = '200 OK'

    start_response(status, response_headers)

    return [response_body]

def dump(environ):

    response_body = ['%s: %s' % (key, value) for key, value in sorted(environ.items())]

    response_body = '\n'.join(response_body)

    return response_body

def send_command(cmd, start_ms):

    REQUEST_TIMEOUT = 30000

    send_msg = cmd

    ipc_sock = "/run/cmdsrv_sock"

    try:

        context = zmq.Context()

        #print "Connecting to server..." - commented out based on your answer

        socket = context.socket(zmq.DEALER)

        socket.connect ("ipc://%s" % ipc_sock)

        # socket.setsockopt(zmq.SOCKOPT_LINGER, 0) this should be the default for close() and is not in python binding

        poll = zmq.Poller()

        poll.register(socket, zmq.POLLIN)

        socket.send_string (send_msg)

        socks = dict(poll.poll(REQUEST_TIMEOUT))

        if socket in socks and socks[socket] == zmq.POLLIN:

            reply_msg = socket.recv()

            if '"Error":' not in reply_msg:

                cur_ms = time.time()

                elapsed_ms = cur_ms - start_ms

                reply_msg = '{"Data": ' + reply_msg + ',"Resp_time": "' + str(elapsed_ms) + '"}'

        else:

            reply_msg = '{"Error": "No Response from CMDSRV in ' + str(REQUEST_TIMEOUT) + ' milliseconds"' + alert_param

        socket.setsockopt(zmq.LINGER, 0)

        socket.close()

        poll.unregister(socket)

    except Exception as e:

        reply_msg = '{"Error": "' + e.message + '"' + alert_param

    return reply_msg

[Graham Dumpleton]

For a start, you should be doing something like:

    try:

        context = zmq.Context()

        #print "Connecting to server..." - commented out based on your answer

        socket = context.socket(zmq.DEALER)

        socket.connect ("ipc://%s" % ipc_sock)

        try:

        # socket.setsockopt(zmq.SOCKOPT_LINGER, 0) this should be the default for close() and is not in python binding

        poll = zmq.Poller()

        poll.register(socket, zmq.POLLIN)

        socket.send_string (send_msg)

        socks = dict(poll.poll(REQUEST_TIMEOUT))

        if socket in socks and socks[socket] == zmq.POLLIN:

            reply_msg = socket.recv()

            if '"Error":' not in reply_msg:

                cur_ms = time.time()

                elapsed_ms = cur_ms - start_ms

                reply_msg = '{"Data": ' + reply_msg + ',"Resp_time": "' + str(elapsed_ms) + '"}'

        else:

            reply_msg = '{"Error": "No Response from CMDSRV in ' + str(REQUEST_TIMEOUT) + ' milliseconds"' + alert_param

        socket.setsockopt(zmq.LINGER, 0)

finally:

        socket.close()

        try:

            poll.unregister(socket)

        except Exception:

            pass

    except Exception as e:

        reply_msg = '{"Error": "' + e.message + '"' + alert_param

This way if an error occurs and exception passed back, that you ensure you close the socket promptly, and potentially deregister it from the poller if necessary.

If don't do that for errors, socket may stay open.

Graham

You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.

[Tim Moody]

Thanks. I'll try your suggestion.

To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscribe@googlegroups.com.

To post to this group, send email to mod...@googlegroups.com.
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.