Logging from worker threads

[vbs]

I have written a small python server that works as a job queue for different test devices. Each device as a queue of test jobs consisting of robotframework tests. Each queue is executed on a separate thread using robot.run. Is it supported to execute multiple robot.run instances threaded in parallel?

A general problem I stumbled upon:

The generated log outputs for a test run are very sparse. There are missing all (?) outputs I generate in my keywords using the logger. I found this code in robot/output/librarylogger.py:

LOGGING_THREADS = ('MainThread', 'RobotFrameworkTimeoutThread')


def write(msg, level, html=False):
    # Callable messages allow lazy logging internally, but we don't want to
    # expose this functionality publicly. See the following issue for details:
    # https://github.com/robotframework/robotframework/issues/1505
    if callable(msg):
        msg = unic(msg)
    if level.upper() not in ('TRACE', 'DEBUG', 'INFO', 'HTML', 'WARN', 'ERROR'):
        raise DataError("Invalid log level '%s'." % level)
    if threading.currentThread().getName() in LOGGING_THREADS:
        LOGGER.log_message(Message(msg, level, html))

So it looks like logging is only supported from main thread and a thread named 'RobotFrameworkTimeoutThread'. What could I do to enable logging from my worker threads or is it generally a bad idea in the first place?

My current idea is that each worker thread adds its own name to the variable librarylogger.LOGGING_THREADS. Would that be ok?

[Vinicius Kwiecien Ruoso]

Have you take a look at robotbackgroundlogger [1]?

[1] https://github.com/robotframework/robotbackgroundlogger

> --
> You received this message because you are subscribed to the Google Groups
> "robotframework-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to robotframework-u...@googlegroups.com.
> To post to this group, send email to robotframe...@googlegroups.com.
> Visit this group at https://groups.google.com/group/robotframework-users.
> For more options, visit https://groups.google.com/d/optout.

[vbs]

Thanks for answer!

Well no, I did not know that. This could probably solve my problem but I would have to touch all my test libraries (> 15) to make use of it. Also, if I understand it correctly, it is meant to be used in a scenario where you run robot.run from a single thread and your test libraries spawn threads. In my case my test libraries are regular keyword libraries that do not spawn threads so I would rather not modify them for that. If possible I would like to solve it once in the job queue server.

In my case, I invoke robot.run multiple times from different threads in a single python application. So I think if multiple robot.run-calls are able to run independently from each other on different threads (in a single process) then also logging from each thread should technically be possible, no?

If that multi-threaded approach is not ok then I would have to change it in a way that I spawn a new process every time I want to run robot.run.

[Vinicius Kwiecien Ruoso]

You are correct. As for your test libraries, if they already use robot
log API, you can simply change the logger import as the API is
virtually the same. That would not be a very clean solution because
the library itself have information that it will be executed on a
different tread than the main one.

There might be a way to change that logging behavior to allow messages
from the thread that started robot. I'm not sure how big of a task
that is. Pekka?

[vbs]

I will try something like this for now:

class Runner:
    def __init__(self, job):
        librarylogger.LOGGING_THREADS += (threading.current_thread().getName(),)

So my Runner (which runs in a separate thread) will "register" its thread as "allowed to log".

Well, probably a bit evil, but lets see... Since LOGGING_THREADS is not private (has no leading underscore) I feel tempted to touch it. :)

Of course I would really a better solution.

Thanks!

Am Donnerstag, 22. September 2016 13:18:42 UTC+2 schrieb vbs: