How to write to /var/log/linuxcnc.log with Python?

[Yishin Li]

Is it possoble to write to /var/log/linuxcnc.log with Python?
i.e. the Python version of rtapi_print_msg().

Thanks in advance,

-Yishin

[Michael Haberler]

> Am 15.07.2015 um 15:42 schrieb Yishin Li <ys...@araisrobo.com>:
>
> Is it possoble to write to /var/log/linuxcnc.log with Python?
> i.e. the Python version of rtapi_print_msg().

yes, the cython bindings have something like that, see:

https://github.com/machinekit/machinekit/blob/master/src/hal/cython/machinekit/rtapi.pyx#L92-L122

from machinekit import rtapi

log = RTAPILogger(level=rtapi.MSG_ERR,tag="marker")
print >> log, "some message",


in case you are extensively using Python on HAL+RTAPI, I'd recommend glancing through nosetests/*.py - lots of Cython usage examples there

(just weaseling around the fact those are not documented ;)


- Michael


>
> Thanks in advance,
>
> -Yishin
>
> --
> website: http://www.machinekit.io blog: http://blog.machinekit.io github: https://github.com/machinekit
> ---
> You received this message because you are subscribed to the Google Groups "Machinekit" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to machinekit+...@googlegroups.com.
> Visit this group at http://groups.google.com/group/machinekit.
> For more options, visit https://groups.google.com/d/optout.

[Dave Cole]

On 7/15/2015 9:42 AM, Yishin Li wrote:
> /var/log/linuxcnc.log with Python

Try this:
http://www.linuxquestions.org/questions/linux-newbie-8/logging-a-to-var-log-messages-using-python-870951/

Dave

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

[Michael Haberler]

> Am 15.07.2015 um 17:25 schrieb Dave Cole <linuxc...@gmail.com>:
>
> On 7/15/2015 9:42 AM, Yishin Li wrote:
>> /var/log/linuxcnc.log with Python
>
> Try this:
> http://www.linuxquestions.org/questions/linux-newbie-8/logging-a-to-var-log-messages-using-python-870951/

well that writes to syslog, true

HOWEVER it does not log in sequence with the other events in HAL/RTAPI and that can be key for determining causality

that is the primary reason for the unified logging architecture which funnels every log message through a message queue regardless of origin so sequencing is retained - RT, user (remember logging being a mess :-? like a bit here, a bit there, and a bit in dmesg and _no_ sequencing? )

if you dont care about sequencing for one reason or the other, then the Python syslog package is fine

>
> Dave
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>

[Yishin Li]

Hi Michael,

On Thursday, July 16, 2015 at 12:17:06 AM UTC+8, Michael Haberler wrote:

HOWEVER it does not log in sequence with the other events in HAL/RTAPI and that can be key for determining causality

that is the primary reason for the unified logging architecture which funnels every log message through a message queue regardless of origin so sequencing is retained - RT, user (remember logging being a mess :-? like a bit here, a bit there, and a bit in dmesg and _no_ sequencing? )

if you dont care about sequencing for one reason or the other, then the Python syslog package is fine

RTAPILogger works!

Here's the RTAPILogger example:

from machinekit import rtapi
rtapi.init_RTAPI()
log = rtapi.RTAPILogger(level=rtapi.MSG_ERR,tag="marker")
print >> log, "some message"

Here's the /var/log/linuxcnc.log:

Jul 16 10:34:17 interlaken msgd:0: marker:10350:user some message

Awesome!

Thank you very much,

-Yishin

[Alexander Rössler]

Yishin Li writes:

> Hi Michael,
>
> On Thursday, July 16, 2015 at 12:17:06 AM UTC+8, Michael Haberler wrote:
>>
>>
>> HOWEVER it does not log in sequence with the other events in HAL/RTAPI and
>> that can be key for determining causality
>>
>> that is the primary reason for the unified logging architecture which
>> funnels every log message through a message queue regardless of origin so
>> sequencing is retained - RT, user (remember logging being a mess :-? like a
>> bit here, a bit there, and a bit in dmesg and _no_ sequencing? )
>>
>> if you dont care about sequencing for one reason or the other, then the
>> Python syslog package is fine
>>
>>
> RTAPILogger works!
>
> Here's the RTAPILogger example:
> from machinekit import rtapi
> rtapi.init_RTAPI()

No need to call init_RTAPI for logging -> it does only initialize RTAPICommand so it
can be used from the module e.g. rtapi.loadrt('blbla')

> log = rtapi.RTAPILogger(level=rtapi.MSG_ERR,tag="marker")
> print >> log, "some message"
> Here's the /var/log/linuxcnc.log:
> Jul 16 10:34:17 interlaken msgd:0: marker:10350:user some message
>
> Awesome!
>
> Thank you very much,
>
> -Yishin

--
Alexander

[jo...@pocketnc.com]

Unlike rtapi_print_msg, this doesn't seem to append messages to the linuxcnc.error_channel. Is there anyway to do so in python?

[ce...@tuta.io]

Hi,

Jul 24, 2020, 23:14 by jo...@pocketnc.com:

> Unlike rtapi_print_msg, this doesn't seem to append messages to the linuxcnc.error_channel. Is there anyway to do so in python?
>

maybe I don't understand the question (and this python stuff is a little bit outside my comfort zone), however isn't the RTAPILogger implemented as a Cython wrapper of rtapi_print_msg? (In src/hal/cython/machinekit/rtapi.pyx.) And as such it should have the same output.

Of course like many things in Machinekit, the logging project was started, implemented to a functioning state but then unfortunately left without much support and bugfixing, so there are problems. For example https://github.com/machinekit/machinekit-hal/issues/199

Cern.

> --
> website: > http://www.machinekit.io> blog: > http://blog.machinekit.io> github: > https://github.com/machinekit
> ---
> You received this message because you are subscribed to the Google Groups "Machinekit" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to > machinekit+...@googlegroups.com> .

> To view this discussion on the web visit > https://groups.google.com/d/msgid/machinekit/d3244a63-a003-4fa3-9ec0-2c44f1a0543an%40googlegroups.com <https://groups.google.com/d/msgid/machinekit/d3244a63-a003-4fa3-9ec0-2c44f1a0543an%40googlegroups.com?utm_medium=email&utm_source=footer>> .
>

[John Allwine]

I’ll have to dig a little deeper. I’m monitoring the error channel in Python using something like this (this is the only process calling poll on the error_channel):

import linuxcnc
e = linuxcnc.error_channel()

while True:
error = e.poll()
if error:
print error
sleep(.1)

I don’t see error messages come through when I use a separate Python process to log messages using RTAPILogger, but I do when using rtapi_print_msg in a real time component. I wasn’t aware of the debugging issues, so I’ll take a closer look. Thanks for the info!


> On Jul 24, 2020, at 5:03 PM, ce...@tuta.io wrote:
>
> Hi,

[ce...@tuta.io]

What about the ZMQ socket on the msgd process, can you see the messages there? Given that I cannot find the error_channel() definition in Machinekit-HAL, I am not sure if the support for it is done in rtapi logging. I have a mounting anxiety that it is one of the things which were in LinuxCNC but were superseded in Machinekit and now it is a bloody stump. (In other words, something which on some nice day somebody will have to solve/repair).

However, still it is odd that C call to rtapi_print_msg you can see and python call to RTAPILogger you cannot - when it uses the exact same function - or at least it looks like it. As I said, I don't know Python well, but can you stop/step into (debug) when the python calls the rtapi_print_msg?

BTW, are you watching with right logging level?

Cern.


Jul 25, 2020, 02:27 by jo...@pocketnc.com:

> I’ll have to dig a little deeper. I’m monitoring the error channel in Python using something like this (this is the only process calling poll on the error_channel):
>
> import linuxcnc
> e = linuxcnc.error_channel()
>
> while True:
> error = e.poll()
> if error:
> print error
> sleep(.1)
>
> I don’t see error messages come through when I use a separate Python process to log messages using RTAPILogger, but I do when using rtapi_print_msg in a real time component. I wasn’t aware of the debugging issues, so I’ll take a closer look. Thanks for the info!
>
>
>> On Jul 24, 2020, at 5:03 PM, ce...@tuta.io wrote:
>>

>> Hi,

> To view this discussion on the web visit https://groups.google.com/d/msgid/machinekit/EC8705B1-4EE5-4F9E-896A-48C4446AB8A5%40pocketnc.com.
>

[John Allwine]

What's the easiest way to connect directly to the msgd socket?

[ce...@tuta.io]

When I was playing with it some year or two back, I just looked at exported mDNS/DNS-SD/Bonjour services (there is a ton of programs for it on any platform), got the address of the socket and then simply used quick'n'dirty program written by the simple tutorial on zeromq.org. I am not sure if the logs are sent as a plain-text or encoded in Protocol Buffers messages. I think it is plaint-text, but if not, you would then need to decode it.

I was doing it this way because I was using C#. But for Python there is PyMachinetalk: https://github.com/machinekit/pymachinetalk (But I have no idea if and how it works).

Cern.


Jul 27, 2020, 16:00 by jo...@pocketnc.com:

> What's the easiest way to connect directly to the msgd socket?
>
> On Fri, Jul 24, 2020 at 6:54 PM <> ce...@tuta.io> > wrote:
>

>> >>> To unsubscribe from this group and stop receiving emails from it, send an email to > >> machinekit+...@googlegroups.com <mailto:machinekit%2Bunsu...@googlegroups.com>>> > .

>> >>> To view this discussion on the web visit > >> https://groups.google.com/d/msgid/machinekit/d3244a63-a003-4fa3-9ec0-2c44f1a0543an%40googlegroups.com>> <>> https://groups.google.com/d/msgid/machinekit/d3244a63-a003-4fa3-9ec0-2c44f1a0543an%40googlegroups.com?utm_medium=email&utm_source=footer>> >> .
>> >>>
>> >
>> > --
>> > website: >> http://www.machinekit.io>> blog: >> http://blog.machinekit.io>> github: >> https://github.com/machinekit
>> > ---
>> > You received this message because you are subscribed to the Google Groups "Machinekit" group.

>> > To unsubscribe from this group and stop receiving emails from it, send an email to >> machinekit+...@googlegroups.com <mailto:machinekit%2Bunsu...@googlegroups.com>>> .
>> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/machinekit/EC8705B1-4EE5-4F9E-896A-48C4446AB8A5%40pocketnc.com>> .

>> >
>>
>>
>
>
>
> --
> website: > http://www.machinekit.io> blog: > http://blog.machinekit.io> github: > https://github.com/machinekit
> ---
> You received this message because you are subscribed to the Google Groups "Machinekit" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to > machinekit+...@googlegroups.com> .

> To view this discussion on the web visit > https://groups.google.com/d/msgid/machinekit/CAPEK9qZ4%3DP9QOWLLsqJVg%3DdQx2gBg6MaSG%3DwgOE2-H0b1Zu0cA%40mail.gmail.com <https://groups.google.com/d/msgid/machinekit/CAPEK9qZ4%3DP9QOWLLsqJVg%3DdQx2gBg6MaSG%3DwgOE2-H0b1Zu0cA%40mail.gmail.com?utm_medium=email&utm_source=footer>> .
>

[John Allwine]

Thanks for all the info, Cern!

I dug a little deeper and on my Beaglebone that has a prebuilt MachineKit image from July of 2019 does not log rtapi_print_msg(RTAPI_MSG_ERR, ... ) calls to the msgd socket, nor the linuxcnc.error_channel. I also have another Beaglebone with a RIP install built from source back in March of this year. It behaves a little differently, but I haven't investigated too far. I know that some of my rtapi_print_msg calls were getting picked up by linuxcnc.error_channel and given there's work in progress to address logging issues, I'm not all that surprised by the difference in behavior. I'll let things shake out a little more before going further, but I appreciate all the help!

In case it helps anyone, I was able to connect to the msgd socket and did have to decode the messages in python like so:

import zmq 

import time 

from machinetalk.protobuf.message_pb2 import Container 

from machinetalk.protobuf.types_pb2 import * 

context = zmq.Context() 

sub = context.socket(zmq.SUB) 

sub.setsockopt(zmq.SUBSCRIBE, "") 

sub.connect("ipc:///tmp/0.log.<MACHINEKIT MKUUID>") 

rx = Container()

while True: 

  msg = sub.recv() 

  rx.ParseFromString(msg)

  print("Received message: %s" % (rx,))

[ce...@tuta.io]

Jul 28, 2020, 22:23 by jo...@pocketnc.com:

> Thanks for all the info, Cern!
>
> I dug a little deeper and on my Beaglebone that has a prebuilt MachineKit image from July of 2019 does not log rtapi_print_msg(RTAPI_MSG_ERR, ... ) calls to the msgd socket, nor the linuxcnc.error_channel. I also have another Beaglebone with a RIP install built from source back in March of this year. It behaves a little differently, but I haven't investigated too far. I know that some of my rtapi_print_msg calls were getting picked up by linuxcnc.error_channel and given there's work in progress to address logging issues, I'm not all that surprised by the difference in behavior. I'll let things shake out a little more before going further, but I appreciate all the help!
>

I recently shallowly investigated the logging because I was interested in what pats would need to be changed to kick the RTAPI/ULAPI compiler switches down to the trashbin of history. The logger is based on Simon Kelley's syslog_async code from http://www.thekelleys.org.uk/syslog-async/ . But time moves ever forward and today there are nice libraries for quick logging like  https://github.com/RafaGago/mini-async-log-c , https://github.com/HardySimpson/zlog , https://github.com/zma/zlog/ and many others. None or very few of these are implemented with real-time systems in mind, so there is always work associated.

One of the reasons why I looked at it is because of the speed of logging, respective what is done on the producer side (i.e. in real-time thread) and what is done on the consumer side. (With rt preempt it is not going to be an issue, given the minimal latency, but with EVL Core I have been able to reach very nice lattencies where the logging speed and load will be important). But I haven't done any real testings yet.

However, all this being said - to be honest - in the set of things which must be done and issues needing repair in Machinekit, I consider this pretty low priority. (Sorry.) Any patches will be welcomed by me, and I am sure by other members of Machinekit community.

Cern.

>
> In case it helps anyone, I was able to connect to the msgd socket and did have to decode the messages in python like so:
> import zmq 
> import time 
> from machinetalk.protobuf.message_pb2 import Container 
> from machinetalk.protobuf.types_pb2 import * 
>
> context = zmq.Context() 
> sub = context.socket(zmq.SUB) 
> sub.setsockopt(zmq.SUBSCRIBE, "") 
> sub.connect("ipc:///tmp/0.log.<MACHINEKIT MKUUID>") 
> rx = Container()
> while True: 
>   msg = sub.recv() 
>   rx.ParseFromString(msg)
>   print("Received message: %s" % (rx,))
>
> On Tue, Jul 28, 2020 at 2:08 PM <> ce...@tuta.io> > wrote:
>

>> When I was playing with it some year or two back, I just looked at exported mDNS/DNS-SD/Bonjour services (there is a ton of programs for it on any platform), got the address of the socket and then simply used quick'n'dirty program written by the simple tutorial on >> zeromq.org <http://zeromq.org>>> . I am not sure if the logs are sent as a plain-text or encoded in Protocol Buffers messages. I think it is plaint-text, but if not, you would then need to decode it.

>> >>  >>> To unsubscribe from this group and stop receiving emails from it, send an email to > >> >> machinekit+...@googlegroups.com <mailto:machinekit%2Bunsu...@googlegroups.com>>> <mailto:>> machinekit%2Bunsu...@googlegroups.com <mailto:machinekit%252Buns...@googlegroups.com>>> >>> > .

>> >>  >>> To view this discussion on the web visit > >> >> https://groups.google.com/d/msgid/machinekit/d3244a63-a003-4fa3-9ec0-2c44f1a0543an%40googlegroups.com>> >>  <>> >> https://groups.google.com/d/msgid/machinekit/d3244a63-a003-4fa3-9ec0-2c44f1a0543an%40googlegroups.com?utm_medium=email&utm_source=footer>> >> >> .
>> >>  >>>
>> >>  >
>> >>  > --
>> >>  > website: >> >> http://www.machinekit.io>> >>  blog: >> >> http://blog.machinekit.io>> >>  github: >> >> https://github.com/machinekit
>> >>  > ---
>> >>  > You received this message because you are subscribed to the Google Groups "Machinekit" group.

>> >>  > To unsubscribe from this group and stop receiving emails from it, send an email to >> >> machinekit+...@googlegroups.com <mailto:machinekit%2Bunsu...@googlegroups.com>>> <mailto:>> machinekit%2Bunsu...@googlegroups.com <mailto:machinekit%252Buns...@googlegroups.com>>> >>> .

>> >>  > To view this discussion on the web visit >> >> https://groups.google.com/d/msgid/machinekit/EC8705B1-4EE5-4F9E-896A-48C4446AB8A5%40pocketnc.com>> >> .
>> >>  >
>> >> 
>> >>
>> >
>> >
>> >
>> > --
>> >  website: > >> http://www.machinekit.io>> >  blog: > >> http://blog.machinekit.io>> >  github: > >> https://github.com/machinekit
>> >  ---
>> >  You received this message because you are subscribed to the Google Groups "Machinekit" group.

>> >  To unsubscribe from this group and stop receiving emails from it, send an email to > >> machinekit+...@googlegroups.com <mailto:machinekit%2Bunsu...@googlegroups.com>>> > .

>> >  To view this discussion on the web visit > >> https://groups.google.com/d/msgid/machinekit/CAPEK9qZ4%3DP9QOWLLsqJVg%3DdQx2gBg6MaSG%3DwgOE2-H0b1Zu0cA%40mail.gmail.com>> <>> https://groups.google.com/d/msgid/machinekit/CAPEK9qZ4%3DP9QOWLLsqJVg%3DdQx2gBg6MaSG%3DwgOE2-H0b1Zu0cA%40mail.gmail.com?utm_medium=email&utm_source=footer>> >> .

>> >
>>
>>
>
>
>
> --
> website: > http://www.machinekit.io> blog: > http://blog.machinekit.io> github: > https://github.com/machinekit
> ---
> You received this message because you are subscribed to the Google Groups "Machinekit" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to > machinekit+...@googlegroups.com> .

> To view this discussion on the web visit > https://groups.google.com/d/msgid/machinekit/CAPEK9qaOtMo-7yhU9Noeu0uKFap-Ew5Z-4nMJb0qGO6i7tqZ%2BQ%40mail.gmail.com <https://groups.google.com/d/msgid/machinekit/CAPEK9qaOtMo-7yhU9Noeu0uKFap-Ew5Z-4nMJb0qGO6i7tqZ%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>> .
>

[John Allwine]

I agree it’s not a high priority and if it becomes one for us, I’ll certainly contribute where I am able.

I still need to get my pull request in order for getting hal_pru_generic working on the BeagleBone AI, as well as making it an instantiable component to be able to leverage multiple PRUs. It’s working, but needs some polish first.

> On Jul 29, 2020, at 5:09 PM, ce...@tuta.io wrote:
>
> Jul 28, 2020, 22:23 by jo...@pocketnc.com: