Getting Syslog working on OSX Monterey

[Philip Bloom]

Hello,

First time mailing and looking for help/guidance. Hopefully not too in the
wrong place.

We've had an app with logging to a /var/log for many years through
logging.sysloghandler. Recently, though, I noticed that it suddenly was
getting no logs whatsoever over there and investigated, believing the
recent triggering change was upgrading to Mac OSX 12 (Monterey).

My understanding is a bit primitive, but our config had never been hard to
follow previously and I may be missing something fundamental. I've looked
over a number of online examples but they don't seem to be running
correctly either. I'm on Python 3.6.8 (though we're about to start an
upgrade to the latest stable, if that in theory may have changes to syslog
handling). I'm mostly ending up here since I'm finding such differences in
response between python modules I'd expect to be fairly similar.

Separating into a test script I've been testing with logging.SysLogHandler,
syslog, and the command line 'syslog -s'. Replaced all below with a
placeholder appName but it's normally a fairly unique setupCDNGUI name.

Example with syslog (python module): - This will show up in System.log but
not in Apple's Console for all messages to device, though won't get all the
way to file.
import syslog
syslog.syslog(syslog.LOG_NOTICE, 'AppName: THIS IS A TEST - Processing
started')
syslog.syslog(syslog.LOG_ERR, 'AppName: THIS IS A TEST ERROR - Processing
started')

Example with SyslogHandler: - This doesn't seem to show up anywhere besides
the screen log, which is odd to me as reading the docs, I understood this
to be a logging integration into the syslog module, so expected similar
behavior in at least what was sent out. Syslog handler is definitely
picked up in logging.handlers

root_logger = logging.getLogger()
root_logger.setLevel(logging.DEBUG)
basic_datefmt = '%m/%d/%Y %I:%M:%S %p'

formatter = logging.Formatter(fmt='%(asctime)s %(levelname)s: %(message)s',
datefmt=basic_datefmt)
syslog_format = logging.Formatter(fmt='SetupCDNGUI: %(message)s',
datefmt=basic_datefmt)

scrhandler = logging.StreamHandler()
scrhandler.setFormatter(formatter)
root_logger.addHandler(scrhandler)

sys_handler = SysLogHandler(address='/var/run/syslog')
#sys_handler.encodePriority(SysLogHandler.LOG_USER, SysLogHandler.LOG_ALERT)
# Tried with the above, but didn't make a difference. Neither did not
defining the address and letting it go to local host.
sys_handler.setFormatter(syslog_format)
root_logger.addHandler(sys_handler)

logging.critical("This is a test")
logging.error("This is a test error")
logging.info("Still a test")
logging.debug("Testy test")

Using command line: Oddly, this gets to Console.Apps messages from device
reliably, though doesn't get picked up by syslog -w or get received by the
ASL config redirect:
syslog -s -l error -k Message "appName: this is a test2"
syslog -s -l notice -k Message "appName: this is a test3"

ASL configuration, which is loaded according to syslog -config:
> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
file_max=50M all_max=500M
? [CA= Sender appName] file /var/log/appName/appName.log

My end goal is really to get just a working python logging ->
var/log/appname/appname.log again so glad to just be pointed in the right
direction if way off base.

--
Philip Bloom
Director, Services Engineering
*AppLovin Corporation*
M: (786)-338-1439 <786-338-1439>
[image: LinkedIn] <https://www.linkedin.com/company/applovin> [image:
Twitter] <https://twitter.com/AppLovin> [image: Facebook]
<https://facebook.com/AppLovin> [image: Instagram]
<https://www.instagram.com/applovin/>
[image: AppLovin] <https://www.applovin.com/>

[Dennis Lee Bieber]

On Sun, 27 Feb 2022 08:34:21 -0800, Philip Bloom
<philip...@applovin.com> declaimed the following:

>
>sys_handler = SysLogHandler(address='/var/run/syslog')

As I read the documentation, the address is supposed to be the NETWORK
address of the machine to receive the log messages...

https://docs.python.org/3/library/logging.handlers.html#sysloghandler

>#sys_handler.encodePriority(SysLogHandler.LOG_USER, SysLogHandler.LOG_ALERT)
># Tried with the above, but didn't make a difference. Neither did not
>defining the address and letting it go to local host.
>sys_handler.setFormatter(syslog_format)
>root_logger.addHandler(sys_handler)
>
>logging.critical("This is a test")
>logging.error("This is a test error")
>logging.info("Still a test")
>logging.debug("Testy test")
>
>Using command line: Oddly, this gets to Console.Apps messages from device
>reliably, though doesn't get picked up by syslog -w or get received by the
>ASL config redirect:
>syslog -s -l error -k Message "appName: this is a test2"
>syslog -s -l notice -k Message "appName: this is a test3"
>
>ASL configuration, which is loaded according to syslog -config:
>> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
>file_max=50M all_max=500M
>? [CA= Sender appName] file /var/log/appName/appName.log
>
>My end goal is really to get just a working python logging ->
>var/log/appname/appname.log again so glad to just be pointed in the right
>direction if way off base.


--

Wulfraed Dennis Lee Bieber AF6VN
wlf...@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

[Dennis Lee Bieber]

On Sun, 27 Feb 2022 14:17:39 -0500, Dennis Lee Bieber
<wlf...@ix.netcom.com> declaimed the following:

APOLOGIES -- I thought I KILLED the draft message, not sent it...

[Philip Bloom]

Thanks. Had tried it with no address, which defaults to ('localhost',
'514') as well as address='/var/run/syslog' which had been working
previously, and the doc recommends as:
For example, on Linux it’s usually ‘/dev/log’ but on OS/X it’s
‘/var/run/syslog’. You’ll need to check your platform and use the
appropriate address (you may need to do this check at runtime if your
application needs to run on several platforms)

Aiming for this to be being picked up on the same computer I'm running the
test script (Python -> Logging -> SyslogHandler -> Mac OS Syslogd -> parsed
ASL config -> /var/log/Appname/Appname.log). It's why I think I may be
missing something fundamental, but it feels like something subtle changed
in the latest OSX.

On Sun, Feb 27, 2022 at 11:26 AM Dennis Lee Bieber <wlf...@ix.netcom.com>
wrote:

> --
> https://mail.python.org/mailman/listinfo/python-list

[Barry]

> On 27 Feb 2022, at 18:36, Philip Bloom via Python-list <pytho...@python.org> wrote:
>
> Hello,

> sys_handler = SysLogHandler(address='/var/run/syslog')

> #sys_handler.encodePriority(SysLogHandler.LOG_USER, SysLogHandler.LOG_ALERT)
> # Tried with the above, but didn't make a difference. Neither did not
> defining the address and letting it go to local host.
> sys_handler.setFormatter(syslog_format)
> root_logger.addHandler(sys_handler)
>
> logging.critical("This is a test")
> logging.error("This is a test error")
> logging.info("Still a test")
> logging.debug("Testy test")
>
> Using command line: Oddly, this gets to Console.Apps messages from device
> reliably, though doesn't get picked up by syslog -w or get received by the
> ASL config redirect:
> syslog -s -l error -k Message "appName: this is a test2"
> syslog -s -l notice -k Message "appName: this is a test3"
>
> ASL configuration, which is loaded according to syslog -config:
>> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
> file_max=50M all_max=500M
> ? [CA= Sender appName] file /var/log/appName/appName.log
>
> My end goal is really to get just a working python logging ->
> var/log/appname/appname.log again so glad to just be pointed in the right
> direction if way off base.

If you look at the code of the logging modules syslog handle you will see that
it does not use syslog. It’s assuming that it can network to a syslog listener.
Such a listener is not running on my systems as far as I know.

I have always assumed that if I want a logger syslog handler that I would have
to implement it myself. So far I have code that uses syslog directly and have
not written that code yet.

Barry

>
> --
> Philip Bloom
> Director, Services Engineering
> *AppLovin Corporation*
> M: (786)-338-1439 <786-338-1439>
> [image: LinkedIn] <https://www.linkedin.com/company/applovin> [image:
> Twitter] <https://twitter.com/AppLovin> [image: Facebook]
> <https://facebook.com/AppLovin> [image: Instagram]
> <https://www.instagram.com/applovin/>
> [image: AppLovin] <https://www.applovin.com/>

> --
> https://mail.python.org/mailman/listinfo/python-list
>

[Peter J. Holzer]

On 2022-02-27 22:16:54 +0000, Barry wrote:
> If you look at the code of the logging modules syslog handle you will see that
> it does not use syslog. It’s assuming that it can network to a syslog listener.
> Such a listener is not running on my systems as far as I know.
>
> I have always assumed that if I want a logger syslog handler that I would have
> to implement it myself. So far I have code that uses syslog directly and have
> not written that code yet.

What do you mean by using syslog directly? The syslog(3) library
function also just sends messages to a "syslog listener" (more commonly
called a syslog daemon) - at least on any unix-like system I'm familiar
with (which doesn't include MacOS). It will, however, always use the
*local* syslog daemon - AFAIK there is no standard way to open a remote
connection (many syslog daemons can be configured to forward messages to
a remote server, however).

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | h...@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

[Barry Scott]

> On 28 Feb 2022, at 21:41, Peter J. Holzer <hjp-p...@hjp.at> wrote:
>
> On 2022-02-27 22:16:54 +0000, Barry wrote:
>> If you look at the code of the logging modules syslog handle you will see that
>> it does not use syslog. It’s assuming that it can network to a syslog listener.
>> Such a listener is not running on my systems as far as I know.
>>
>> I have always assumed that if I want a logger syslog handler that I would have
>> to implement it myself. So far I have code that uses syslog directly and have
>> not written that code yet.
>
> What do you mean by using syslog directly? The syslog(3) library
> function also just sends messages to a "syslog listener" (more commonly
> called a syslog daemon) - at least on any unix-like system I'm familiar
> with (which doesn't include MacOS). It will, however, always use the
> *local* syslog daemon - AFAIK there is no standard way to open a remote
> connection (many syslog daemons can be configured to forward messages to
> a remote server, however).

I'm re-reading the code to check on what I'm seeing. (Its been a long
time since I last look deeply at this code).

You can write to /dev/log if you pass that to
SysLogHandler(address='/dev/log'), but the default is to use a socket
to talk to a network listener on localhost:514. There are no deamons
listening on port 514 on my Fedora systems or mac OS.

That is not what you would expect as the default if you are using the C
API.

What you do not see used in the SyslogHandler() is the import syslog
and hence its nor using openlog() etc from syslog API.

Barry

> hp
>
> --
> _ | Peter J. Holzer | Story must make more sense than reality.
> |_|_) | |
> | | | h...@hjp.at | -- Charles Stross, "Creative writing
> __/ | http://www.hjp.at/ | challenge!"

> --
> https://mail.python.org/mailman/listinfo/python-list

[Philip Bloom]

I'm probably asking on the wrong list, and probably should bother wherever
apple's ASL experts live for changes in monterey. Guess nobody else is
seeing this?

The same exact code is working just fine on OSX Big Sur, but on OSX
Monterey it doesn't work at all. Users that haven't updated are having the
program produce logs as it has for years through
logging.handlers.SysLogHandler. Mac OSX definitely has a listening socket
at '/var/run/syslog' which shows up in Console.app.

Apologies, Barry. I'm not quite understanding your responses.

When we say OSX has no listener for Syslog, what is the Apple System Log
and the general output to Console.app then? I thought that was the local
syslog on OSX and had, for years, been behaving as such when using
logging.handlers.SysLogHandler with a config in /etc/asl/ to define how it
should be routed and the rollover/cleanup frequency.

Thanks for replying, just having trouble understanding.

> --
> https://mail.python.org/mailman/listinfo/python-list

[Barry Scott]

> On 3 Mar 2022, at 03:01, Philip Bloom <philip...@applovin.com> wrote:
>
> I'm probably asking on the wrong list, and probably should bother wherever apple's ASL experts live for changes in monterey. Guess nobody else is seeing this?
>
> The same exact code is working just fine on OSX Big Sur, but on OSX Monterey it doesn't work at all. Users that haven't updated are having the program produce logs as it has for years through logging.handlers.SysLogHandler. Mac OSX definitely has a listening socket at '/var/run/syslog' which shows up in Console.app.
>
> Apologies, Barry. I'm not quite understanding your responses.

This works:

syslog.openlog(logoption=syslog.LOG_PID, facility=syslog.LOG_USER)
syslog.syslog(syslog.LOG_NOTICE, 'QQQ test log')

I see the "QQQ test log" when I run syslog command.

And I can reproduce that logging.handlers.SysLogHandler does not work.
I added debug to the SysLogHandler() code and see that it does indeed write into the /var/run/syslog socket.
I suspect that macOS /var/run/syslog does not implement the RFC that this code depends on, but that a wild guess.

I suggest that you write your own handler that uses syslog.syslog() and use that.

Barry

>
> When we say OSX has no listener for Syslog, what is the Apple System Log and the general output to Console.app then? I thought that was the local syslog on OSX and had, for years, been behaving as such when using logging.handlers.SysLogHandler with a config in /etc/asl/ to define how it should be routed and the rollover/cleanup frequency.
>
> Thanks for replying, just having trouble understanding.
>
>

> On Mon, Feb 28, 2022 at 2:07 PM Barry Scott <ba...@barrys-emacs.org <mailto:ba...@barrys-emacs.org>> wrote:

> > | | | h...@hjp.at <mailto:h...@hjp.at> | -- Charles Stross, "Creative writing
> > __/ | http://www.hjp.at/ <http://www.hjp.at/> | challenge!"
> > --
> > https://mail.python.org/mailman/listinfo/python-list <https://mail.python.org/mailman/listinfo/python-list>
>
> --
> https://mail.python.org/mailman/listinfo/python-list <https://mail.python.org/mailman/listinfo/python-list>

>
>
> --
> Philip Bloom
> Director, Services Engineering

> AppLovin Corporation
> M: (786)-338-1439 <tel:786-338-1439>
> <https://www.linkedin.com/company/applovin> <https://twitter.com/AppLovin> <https://facebook.com/AppLovin> <https://www.instagram.com/applovin/>
> <https://www.applovin.com/>
>
>
>

[Philip Bloom]

Grabbing latest python that does work. Good we're about to get out of the
stone ages a bit here.

So findings:
Syslog - works in 3.10, broken against monterey in 3.6.
Logging.Handlers.Sysloghandler - is broken in both against Monterey.

Will bug it for the tracker. Thanks for the feedback.

>> > | | | h...@hjp.at | -- Charles Stross, "Creative writing
>> > __/ | http://www.hjp.at/ | challenge!"
>> > --

>> > https://mail.python.org/mailman/listinfo/python-list
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
>

> --
> Philip Bloom
> Director, Services Engineering

> *AppLovin Corporation*
> M: (786)-338-1439 <786-338-1439>
> [image: LinkedIn] <https://www.linkedin.com/company/applovin> [image:
> Twitter] <https://twitter.com/AppLovin> [image: Facebook]
> <https://facebook.com/AppLovin> [image: Instagram]
> <https://www.instagram.com/applovin/>
> [image: AppLovin] <https://www.applovin.com/>
>
>
>
>
>
>

--
Philip Bloom
Director, Services Engineering

[Peter J. Holzer]

On 2022-02-28 22:05:05 +0000, Barry Scott wrote:
> On 28 Feb 2022, at 21:41, Peter J. Holzer <hjp-p...@hjp.at> wrote:
> > On 2022-02-27 22:16:54 +0000, Barry wrote:
> >> I have always assumed that if I want a logger syslog handler that I would have
> >> to implement it myself. So far I have code that uses syslog directly and have
> >> not written that code yet.
> >
> > What do you mean by using syslog directly? The syslog(3) library
> > function also just sends messages to a "syslog listener" (more commonly
> > called a syslog daemon) - at least on any unix-like system I'm familiar
> > with (which doesn't include MacOS). It will, however, always use the
> > *local* syslog daemon - AFAIK there is no standard way to open a remote
> > connection (many syslog daemons can be configured to forward messages to
> > a remote server, however).
>
> I'm re-reading the code to check on what I'm seeing. (Its been a long
> time since I last look deeply at this code).
>
> You can write to /dev/log if you pass that to
> SysLogHandler(address='/dev/log'), but the default is to use a socket
> to talk to a network listener on localhost:514. There are no deamons
> listening on port 514 on my Fedora systems or mac OS.

If you are saying that SysLogHandler should use a system specific
default (e.g. "/dev/log" on Linux) instead of UDP port 514 everywhere, I
agree 99 % (the remaining 1 % is my contrarian alter ego arguing that
that's really the distribution maintainer's job since a Linux
distribution might use some other socket).

If you are saying it should use the libc syslog routines, I disagree for
at least two reasons: a) they are OS specific, b) you can't configure the
destination. So that would remove useful functionality.

In any case it seems strange to me that you want to rewrite it just to
avoid passing a single parameter to the constructor (or - more likely -
adding a single line to a config file).

> What you do not see used in the SyslogHandler() is the import syslog
> and hence its nor using openlog() etc from syslog API.

For good reasons. The C syslog API is missing important functionality.

[Barry Scott]

What I am used to is a setup where programs use syslog() to log and
a dameon like rsyslog is responsible for routing the logs to local files
and/or remote systems. On a modern linux I think the route is
syslog() -> journald. And you can have syslog() -> journald -> rsyslogd.

Using the syslog() function means that any platform/distro details are
hidden from the user of syslog() and as is the case of macOS it
"just works". (I assume, not checked, that the write to the socket does not work
because Apple is not implementing the syslog protocol from the RFC).

>
>> What you do not see used in the SyslogHandler() is the import syslog
>> and hence its nor using openlog() etc from syslog API.
>
> For good reasons. The C syslog API is missing important functionality.

What are you think about being missing? Just curious.

As an aside if I had the need to log into a system log mechanism I'd be
looking to use the jounald API so that I can use structured logging on linux
systems.

Barry

>
> hp
>
> --
> _ | Peter J. Holzer | Story must make more sense than reality.
> |_|_) | |
> | | | h...@hjp.at | -- Charles Stross, "Creative writing
> __/ | http://www.hjp.at/ | challenge!"

> --
> https://mail.python.org/mailman/listinfo/python-list

[Peter J. Holzer]

On 2022-03-05 16:25:38 +0000, Barry Scott wrote:
> On 4 Mar 2022, at 21:23, Peter J. Holzer <hjp-p...@hjp.at> wrote:
> > If you are saying that SysLogHandler should use a system specific
> > default (e.g. "/dev/log" on Linux) instead of UDP port 514 everywhere, I
> > agree 99 % (the remaining 1 % is my contrarian alter ego arguing that
> > that's really the distribution maintainer's job since a Linux
> > distribution might use some other socket).
> >
> > If you are saying it should use the libc syslog routines, I disagree for
> > at least two reasons: a) they are OS specific, b) you can't configure the
> > destination. So that would remove useful functionality.
> >
> > In any case it seems strange to me that you want to rewrite it just to
> > avoid passing a single parameter to the constructor (or - more likely -
> > adding a single line to a config file).
>
> What I am used to is a setup where programs use syslog() to log and
> a dameon like rsyslog is responsible for routing the logs to local files
> and/or remote systems.

Yes, that's a pretty common setup.

> On a modern linux I think the route is syslog() -> journald. And you
> can have syslog() -> journald -> rsyslogd.

Yes. Journald accepts syslog (RFC 5424) protocol on /dev/log and can
optionally forward messages to another syslog daemon (but only via Unix
sockets, AFAICS, not via UDP or TCP). rsyslog is then configured to
accept those forwarded messages (and may also be configured to accept
messages on port 514, but this isn't the case by default on Debian or
Ubuntu).

> Using the syslog() function means that any platform/distro details are
> hidden from the user of syslog() and as is the case of macOS it
> "just works".

That doesn't seem to be case. Philip Bloom reported in
<CADoNktVgU9DBdBfc5wJe4vhJcTKrdVQjdeUO8eN=BvTw9...@mail.gmail.com>,
that syslog in Python 3.6 broke in OS X Monterey. So even using
syslog(3) doesn't seem to be safe across upgrades (no, I don't know
how Apple managed to break this - as I wrote, I don't use OS X).

> (I assume, not checked, that the write to the socket does not work
> because Apple is not implementing the syslog protocol from the RFC).
>
> >
> >> What you do not see used in the SyslogHandler() is the import syslog
> >> and hence its nor using openlog() etc from syslog API.
> >
> > For good reasons. The C syslog API is missing important functionality.
>
> What are you think about being missing? Just curious.

As I already wrote:

* System independence. The libc syslog(3) function (and the Python
syslog package using it) exists only on Unix-like OSs. Not Windows.
Probably not some embedded platforms. SyslogHandler works on all
platforms with networking.
* Configurable targets. The SyslogHandler allows you to specify where to
send the messages. syslog(3) does not. So with syslog(3) you always
need a locally running daemon. In containers or embedded systems this
might add extra unnecessary complexity.

> As an aside if I had the need to log into a system log mechanism I'd be
> looking to use the jounald API so that I can use structured logging on linux
> systems.

That might be useful. Journald already adds a bit of extra information
to messages received over the syslog, but using the API directly would
probably give the user more control.

[Peter J. Holzer]

On 2022-03-06 18:28:59 +0100, Peter J. Holzer wrote:
> On 2022-03-05 16:25:38 +0000, Barry Scott wrote:
> > Using the syslog() function means that any platform/distro details are
> > hidden from the user of syslog() and as is the case of macOS it
> > "just works".
>
> That doesn't seem to be case. Philip Bloom reported in
> <CADoNktVgU9DBdBfc5wJe4vhJcTKrdVQjdeUO8eN=BvTw9...@mail.gmail.com>,
> that syslog in Python 3.6 broke in OS X Monterey. So even using
> syslog(3) doesn't seem to be safe across upgrades (no, I don't know
> how Apple managed to break this - as I wrote, I don't use OS X).
>
>
> > (I assume, not checked, that the write to the socket does not work
> > because Apple is not implementing the syslog protocol from the RFC).

I noticed that RFC 5424 describes a very different protocol from RFC
3164 and was curious which one SysLogHandler implements.

Surprise: Neither.

It just sends the PRI part and the MSG part. While that neatly sidesteps
the question of which of the two incompatible header formats to use,
it's not a a valid syslog message in either format.

Obviously Linux accepts this format (the timestamp and the hostname are
kind of redundant anyway when you're using a unix socket).

I strongly suspect that the OS X syslog daemon got stricter and now
insists on properly formatted messages.

[Barry]

> On 6 Mar 2022, at 19:38, Peter J. Holzer <hjp-p...@hjp.at> wrote:

>
> On 2022-03-06 18:28:59 +0100, Peter J. Holzer wrote:
>>> On 2022-03-05 16:25:38 +0000, Barry Scott wrote:
>>> Using the syslog() function means that any platform/distro details are
>>> hidden from the user of syslog() and as is the case of macOS it
>>> "just works".
>>
>> That doesn't seem to be case. Philip Bloom reported in
>> <CADoNktVgU9DBdBfc5wJe4vhJcTKrdVQjdeUO8eN=BvTw9...@mail.gmail.com>,
>> that syslog in Python 3.6 broke in OS X Monterey. So even using
>> syslog(3) doesn't seem to be safe across upgrades (no, I don't know
>> how Apple managed to break this - as I wrote, I don't use OS X).
>>
>>
>>> (I assume, not checked, that the write to the socket does not work
>>> because Apple is not implementing the syslog protocol from the RFC).
>
> I noticed that RFC 5424 describes a very different protocol from RFC
> 3164 and was curious which one SysLogHandler implements.
>
> Surprise: Neither.

So the logging code works by accident…

>
> It just sends the PRI part and the MSG part. While that neatly sidesteps
> the question of which of the two incompatible header formats to use,
> it's not a a valid syslog message in either format.
>
> Obviously Linux accepts this format (the timestamp and the hostname are
> kind of redundant anyway when you're using a unix socket).
>
> I strongly suspect that the OS X syslog daemon got stricter and now
> insists on properly formatted messages.

Interesting. If I can find a spare hour I can play with this.

Barry

>
> hp
>
>
> --
> _ | Peter J. Holzer | Story must make more sense than reality.
> |_|_) | |
> | | | h...@hjp.at | -- Charles Stross, "Creative writing
> __/ | http://www.hjp.at/ | challenge!"

> --
> https://mail.python.org/mailman/listinfo/python-list