Marathon logging to /var/log/mesos instead of /var/log/<syslog or messages> directory.

4,240 views
Skip to first unread message

Sarjeet Singh

unread,
Aug 25, 2015, 9:14:35 PM8/25/15
to marathon-framework
I'm wondering if there is a way (other than what I've already tried mentioned below) to get the Marathon logs to custom directory (or /var/log/mesos) for Marathon info/warn/error logs? 

By default, marathon has logging configured to syslog which seems to have other system level logs as well which is making hard to debug any marathon issues by tracing just the marathon specific logs.

Although, I found a link of troubleshooting the marathon issues <http://open.mesosphere.com/intro-course/ex19.html>, which seems to do what I was looking for, but it doesn't work in my case. (because of I have marathon running as an java app using the marathon jar). Here is the command:

[root@qa101-137 mesos]# ps -ef | grep 24463

root     13719 12299  0 18:09 pts/0    00:00:00 grep 24463

mapr     24463     1 12 16:55 ?        00:09:07 java -Djava.library.path=/usr/local/lib:/usr/lib:/usr/lib64 -Djava.util.logging.SimpleFormatter.format=%2$s%5$s%6$s%n -Xmx512m -cp /usr/bin/marathon mesosphere.marathon.Main --zk zk://10.10.101.137:5181/marathon --master zk://10.10.101.137:5181/mesos

mapr     24492 24463  0 16:55 ?        00:00:00 logger -p user.info -t marathon[24463]

mapr     24493 24463  0 16:55 ?        00:00:00 logger -p user.notice -t marathon[24463]


From the above tutorial, Here is what I have tried so far,

I've followed the steps as mentioned and didn't get the logging to /var/log/mesos/marathon.log.

[node1 mesos]# cat /etc/rsyslog.d/mesos.conf 

if $programname == 'marathon' then {

   action(type="omfile" file="/var/log/mesos/marathon.log")

}


[root@qa101-137 mesos]# service rsyslog restart

Shutting down system logger:                               [  OK  ]

Starting system logger:                                    [  OK  ]


[root@qa101-137 mesos]# file /var/lib/mesos/marathon.log

/var/lib/mesos/marathon.log: cannot open `/var/lib/mesos/marathon.log' (No such file or directory)


Marathon was already running, and also tried restarting as well, but still the same issue and logs were still recorded to /var/log/messages.


Can someone point me to any guide or reference to achieve this? I've looked through the open JIRA issues for this, but couldn't find any. Let me know if you need any additional details?


Sarjeet Singh

unread,
Aug 25, 2015, 9:18:42 PM8/25/15
to marathon-framework

[root@qa101-137 mesos]# file /var/log/mesos/marathon.log

/var/log/mesos/marathon.log: cannot open `/var/log/mesos/marathon.log' (No such file or directory)


typo in the previous email, /var/log/mesos instead of /var/lib/mesos. 


On Tuesday, August 25, 2015 at 6:14:35 PM UTC-7, Sarjeet Singh wrote:
I'm wondering if there is a way (other than what I've already tried mentioned below) to get the Marathon logs to custom directory (or /var/log/mesos) for Marathon info/warn/error logs? 

By default, marathon has logging configured to syslog which seems to have other system level logs as well which is making hard to debug any marathon issues by tracing just the marathon specific logs.

Although, I found a link of troubleshooting the marathon issues <http://open.mesosphere.com/intro-course/ex19.html>, which seems to do what I was looking for, but it doesn't work in my case. (because of I have marathon running as an java app using the marathon jar). Here is the command:

[root@qa101-137 mesos]# ps -ef | grep 24463

root     13719 12299  0 18:09 pts/0    00:00:00 grep 24463

mapr     24463     1 12 16:55 ?        00:09:07 java -Djava.library.path=/usr/local/lib:/usr/lib:/usr/lib64 -Djava.util.logging.SimpleFormatter.format=%2$s%5$s%6$s%n -Xmx512m -cp /usr/bin/marathon mesosphere.marathon.Main --zk zk://10.10.101.137:5181/marathon --master zk://10.10.101.137:5181/mesos

mapr     24492 24463  0 16:55 ?        00:00:00 logger -p user.info -t marathon[24463]

mapr     24493 24463  0 16:55 ?        00:00:00 logger -p user.notice -t marathon[24463]


From the above tutorial, Here is what I have tried so far,

I've followed the steps as mentioned and didn't get the logging to /var/log/mesos/marathon.log.

[node1 mesos]# cat /etc/rsyslog.d/mesos.conf 

if $programname == 'marathon' then {

   action(type="omfile" file="/var/log/mesos/marathon.log")

}


[root@qa101-137 mesos]# service rsyslog restart

Shutting down system logger:                               [  OK  ]

Starting system logger:                                    [  OK  ]


[root@qa101-137 mesos]# file /var/log/mesos/marathon.log

/var/log/mesos/marathon.log: cannot open `/var/log/mesos/marathon.log' (No such file or directory)

Sarjeet Singh

unread,
Sep 3, 2015, 12:19:02 PM9/3/15
to marathon-framework
Ping...

Can anybody suggest any viable solution for the problem I posed, if there is such solution exists? Let me know if you need any more details, or if the problem posted is vague/not clear.

-Sarjeet


On Tuesday, August 25, 2015 at 6:14:35 PM UTC-7, Sarjeet Singh wrote:

Teodor Pripoae

unread,
Nov 1, 2015, 9:50:17 PM11/1/15
to marathon-framework
I think rsyslog does not have permission to write to /var/log/mesos-master directory. On my setup this directory was owned by root:root. It needs syslog user or group settings.

On my setup I moved the logs to /var/log/chronos.log, /var/log/marathon.log, etc.

Haripriya Ayyalasomayajula

unread,
Nov 4, 2015, 2:19:22 PM11/4/15
to Sarjeet Singh, marathon-framework
Also, please try checking the syntax you are using in the conf file rsyslog.d/ - which version do you have on your system and the syntax you are following. That was the problem in my case! 

--
You received this message because you are subscribed to the Google Groups "marathon-framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to marathon-framew...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Regards,
Haripriya Ayyalasomayajula 

Reply all
Reply to author
Forward
0 new messages