SNMP and Mobicents SIP Servlets

[pdunkley]

Hello,

I have some questions about using SNMP to monitor the status of
Mobicents SIP Servlets (JBoss version). I am interested in general
health monitoring of MSS, in particular I am interested in being able
to tell when there has been a failure or fail-over when using
clustering.

Ideally, I would like to receive SNMP traps when something happens to
my MSS instance that requires attention. I gather that JOPR can raise
SNMP traps in response to alerts. Is that correct?

If it is the case that events raised as JOPR alerts can be sent as
SNMP traps, it would be helpful to know what JOPR alerts MSS raises.
I can't seem to find anything that tells me about JOPR alerts for
MSS. Can anyone point me to a document that describes them?

In the future I will also want to configure my MSS install using
SNMP. I understand that SNMP support is to be included in the next
Mobicents release, and I gather from some searches that this will be
focused on configuration that can currently be done through the JMX
interface. Is that correct?

Ultimately, I will want to be able to manage my SIP Servlets using
SNMP (including raising traps). Will this be possible after the next
MSS version, or will I need to integrate something like SNMP4J into my
servlets?

Thanks for your help,

Peter

[Jean Deruelle]

Hi Peter

Interesting topic, I was just about to open up a thread to poll the community for its SNMP usage and what they would like to see for SNMP Support in Mobicents as a whole and not just Mobicents Sip Servlets to have a conherent and consistent story across the projects, so if other community members could chime in and give their requirements for SNMP and which version of the protocol (JBoss currently support only v2) that would be the best to make sure newest versions of Mobicents suits your needs.

Also this item is open for contributions if anyone is willing to give a hand (or 2), feel free to shout or mail me privately (if you're shy)

See inline,

On Thu, Jan 13, 2011 at 10:10 PM, pdunkley <peter....@crocodile-rcs.com> wrote:

Hello,

I have some questions about using SNMP to monitor the status of
Mobicents SIP Servlets (JBoss version).  I am interested in general
health monitoring of MSS, in particular I am interested in being able
to tell when there has been a failure or fail-over when using
clustering. 

Just to make sure I get you right, you want a trap per call failed over or just only one general trap that a node failed in the cluster ?

Which parameters are you considering when you say health monitoring ?

 

Ideally, I would like to receive SNMP traps when something happens to
my MSS instance that requires attention.  

 

I would be interesting to hear your requirements have more details about that, but Jopr alerts might be the solution as "requires attention" can vary a lot from one architecture to another. That would help scope the work.

I gather that JOPR can raise SNMP traps in response to alerts.  Is that correct?

That's correct, you can setup jopr to raise alerts when memory or CPU is too low by example or when the maximum number of active sip sessions is above a certain limit. Those alerts can trigger SNMP traps 

See http://docs.redhat.com/docs/en-US/JBoss_Operations_Network/2.4/html-single/Basic_Admin_Guide/index.html#configuring-snmp-notifications

 

If it is the case that events raised as JOPR alerts can be sent as
SNMP traps, it would be helpful to know what JOPR alerts MSS raises.

I can't seem to find anything that tells me about JOPR alerts for
MSS.  Can anyone point me to a document that describes them?

MSS doesn't raise JOPR alerts by itself. This is the other way around, you have to configure Jopr to raise alerts about the things you deem important enough to receive traps about. See http://docs.redhat.com/docs/en-US/JBoss_Operations_Network/2.4/html-single/Basic_Admin_Guide/index.html#configuring-alerts

So you can define alerts to monitor attributes of MSS exposed through Jopr so that when it goes over a threshold will fire an SNMP Trap

 

In the future I will also want to configure my MSS install using
SNMP.  I understand that SNMP support is to be included in the next
Mobicents release, and I gather from some searches that this will be
focused on configuration that can currently be done through the JMX
interface.  Is that correct?

Yes the first step scheduled for issue http://code.google.com/p/mobicents/issues/detail?id=1029, is to actually refactor the current JBoss SNMP Adaptor using SNMP4J to support v3 of the protocol and have a more stable SNMP library tahn JoeSNMP and hook the current MSS Specifics on configuration there, but I'm open to add more to it and would like to know if this is what you expected. This work can be done in incremental steps as we gather requirements from community and customers.

 

Ultimately, I will want to be able to manage my SIP Servlets using
SNMP (including raising traps).  Will this be possible after the next
MSS version, or will I need to integrate something like SNMP4J into my
servlets?

This is the intent but if you could provide a (exhaustive?) list of what you encompass by "manage" that would be great.

 

Thanks for your help,

Thanks to you

Jean 

Peter

[pdunkley]

Hello Jean,

Thanks for the response. See inline.

On Jan 14, 1:39 pm, Jean Deruelle <jean.derue...@gmail.com> wrote:
> Hi Peter
>
> Interesting topic, I was just about to open up a thread to poll the
> community for its SNMP usage and what they would like to see for SNMP
> Support in Mobicents as a whole and not just Mobicents Sip Servlets to have
> a conherent and consistent story across the projects, so if other community
> members could chime in and give their requirements for SNMP and which
> version of the protocol (JBoss currently support only v2) that would be the
> best to make sure newest versions of Mobicents suits your needs.
>
> Also this item is open for contributions if anyone is willing to give a hand
> (or 2), feel free to shout or mail me privately (if you're shy)
>

I will ask around the office to see if anyone has any additional
feedback or information we can give you on what we think is needed
from SNMP support.

My experience has shown that the two versions of SNMP that are needed
are v1 and v3. Version 1 is used in simple, closed, situations where
no security is required. Version 3 is used when security is required.

Version 2 has a flawed security model and, as such, offers little
advantage over v1 while being worth little when security is required.

> Just to make sure I get you right, you want a trap per call failed over or
> just only one general trap that a node failed in the cluster ?
> Which parameters are you considering when you say health monitoring ?
>

Just one trap indicating node failure. A trap per call would destroy
any monitoring system!

> I would be interesting to hear your requirements have more details about
> that, but Jopr alerts might be the solution as "requires attention" can vary
> a lot from one architecture to another. That would help scope the work.
>

It will vary from installation to installation. In general anything
that indicates a problem that recurs on a large scale and/or that does
not automatically recover will need an alarm. My experience with
operators has also indicated that there is a wide range of
requirements in terms of reporting, but in general within telecoms:
* Alarms must have severities. Typically, critical, major, minor,
informational. Critical generally means a significant failure of more
than 50% traffic handling, major a significant failure of less than
50% traffic handling, minor a small failure with a small loss of
traffic handling which will (probably) automatically recover,
informational no loss of traffic handling as of yet but something
(which may contribute to future problems) has occurred. Ideally it
would be possible to choose a severity for each alarm.
* Alarms should be maskable, particularly informational ones. Too
many trivial alarms are a curse as they make real problems hard to
spot. Different operators have different opinions on what is trivial.
* Alarms should be stateful. They are not events. They become active
when the problem starts and become recovered when the problem ends.
You may have events that indicate an alarm has been raised or
recovered, but these events are not the alarms themselves (though the
two are often confused).
* Alarms should be de-bounced. That way a condition that is wavering
around the alarm onset/offset criteria will not flood the monitoring
system.

> Yes the first step scheduled for issuehttp://code.google.com/p/mobicents/issues/detail?id=1029, is to actually

> refactor the current JBoss SNMP Adaptor using SNMP4J to support v3 of the
> protocol and have a more stable SNMP library tahn JoeSNMP and hook the
> current MSS Specifics on configuration there, but I'm open to add more to it
> and would like to know if this is what you expected. This work can be done
> in incremental steps as we gather requirements from community and customers.
>

I would like to use SNMP to configure anything that I can do by other
means. Ideally, the configuration changes would be persistent and
written back out to configuration files.

> This is the intent but if you could provide a (exhaustive?) list of what you
> encompass by "manage" that would be great.
>

I would like to be able to "get" and "set" attributes and tables
within my servlet. This will allow me to have per-servlet
configuration and statistics. I cannot be more specific here as each
servlet will be different.

I would also like to be able to generate SNMP traps from my servlet.

Ideally, if Mobicents has some SNMP configuration/alarm/event model
(either independently or through JOPR) I'd like to be able to have
servlet specific configuration, alarms, and events through that model.

One possibility would be to allow people to fetch sets of metrics
through JOPR over SNMP (if it cannot already do that), and have the
ability for servlets to generate their own bespoke metrics for JOPR
display. JOPR alerts (and therefore SNMP traps) could also be
generated this way.

I hope this is helpful,

Peter

[Jean Deruelle]

I will ask around the office to see if anyone has any additional

feedback or information we can give you on what we think is needed
from SNMP support.

My experience has shown that the two versions of SNMP that are needed
are v1 and v3.  Version 1 is used in simple, closed, situations where
no security is required.  Version 3 is used when security is required.

Version 2 has a flawed security model and, as such, offers little
advantage over v1 while being worth little when security is required.

OK good. upgrading to version 3 will be indeed the first step but I actually just saw the AS Team is moving forward with this See https://issues.jboss.org/browse/JBPAPP-5592

It will also be compatible with AS 5.1 version of JBoss. I will be working with the JBoss AS issue owner on this as we gather feedback to make sure this fits Mobicents needs.

As a side note Jopr on the other side support v3 (and uses SNMP4J under the covers IIRC)

 

> Just to make sure I get you right, you want a trap per call failed over or
> just only one general trap that a node failed in the cluster ?
> Which parameters are you considering when you say health monitoring ?
>

Just one trap indicating node failure.  

That is doable.

 

A trap per call would destroy any monitoring system!

This way people can get to their homes early saying the monitoring system is down :-)

I think all of this is supported by the Jopr Alert + SNMP supports, will double check

 

> Yes the first step scheduled for issuehttp://code.google.com/p/mobicents/issues/detail?id=1029, is to actually

> refactor the current JBoss SNMP Adaptor using SNMP4J to support v3 of the
> protocol and have a more stable SNMP library tahn JoeSNMP and hook the
> current MSS Specifics on configuration there, but I'm open to add more to it
> and would like to know if this is what you expected. This work can be done
> in incremental steps as we gather requirements from community and customers.
>

I would like to use SNMP to configure anything that I can do by other
means.  Ideally, the configuration changes would be persistent and
written back out to configuration files.

Persistency will only be achievable on top of JBoss AS 7.

> This is the intent but if you could provide a (exhaustive?) list of what you
> encompass by "manage" that would be great.
>

I would like to be able to "get" and "set" attributes and tables
within my servlet.  This will allow me to have per-servlet
configuration and statistics.  I cannot be more specific here as each
servlet will be different.

So dynamic registration of SNMP attributes and tables from your servlet ? Through an API (or access to the underlying SNMP4J API) or through an application MIB that would be registered/unregistered by the container automatically on application startup/shutdown ? If you can be more specific on how at usage you would like to register those per servlet SNMP specific attributes it would be nice.

As a side note :This is already possible with the current implementation for JMX (See below) but I guess you would like to remove this JMX layer completely.

(Inside the snmp-agent.sar directory, you'll see a file called attributes.xml. This file associates SNMP OIDs with JMX attributes.
So for each item in your MIB, you need to expose that information via a JMX MBean, and then add the OID->JMX association to attributes.xml.
The agent itself neither knows nor cares about your MIB, so you need to be careful to specify the OIDs in attributes.xml properly.)

I would also like to be able to generate SNMP traps from my servlet.

Ok, will add it as well.

 

Ideally, if Mobicents has some SNMP configuration/alarm/event model
(either independently or through JOPR) I'd like to be able to have
servlet specific configuration, alarms, and events through that model.

One possibility would be to allow people to fetch sets of metrics
through JOPR over SNMP (if it cannot already do that)

I'm not sure you need JOPR for that, I think the set of metrics can already be fetched

 

, and have the
ability for servlets to generate their own bespoke metrics for JOPR
display.  

This will be achievable by exposing JMX metrics and then writing a Jopr Plugin to fetch them and display them in the Jopr Console.

 

JOPR alerts (and therefore SNMP traps) could also be
generated this way.

If you have the set of metrics available through Jopr, yes

 

Another option instead of what is already existing (JMX) might be to write a Jopr Plugin that would fetch the MIB information, and display the metrics by fetching the SNMP metrics directly thus bypassing the need for a JMX layer. Not sure this would be possible yet (I would need to talk to the Jopr guys) but is that something that you would prefer ?

[aayush]

In my opinion, we should first freeze the managed entities for Mobicents -

1. MSS
2. JAIN SLEE
3. Others?

Based on this, we should model standard MIBs for each managed entity. Each Mobicents server would be maintaining metrics and exposing it using MBeans for example.

Hence our MIBs need to tie back closely to these existing MBeans.

The SNMP agent for Mobicents can be deployed as a standalone SAR binary similar to what JBOSS does now and shipped as a common module.

Based on which server is being operated (MSS or SLEE), MIBs can be dropped into a specific folder of the binary and registered with the agent during startup.

These MIBs would behave as "standard mibs" for the mobicents distribution.

The other requirement is for the application (sip servlet or sbb) to raise traps and expose information over SNMP.  MIBs in this case need to be modelled manually and provided to the EMS.

For MIB implementation in JAIN SLEE applications, we can escalate standard SLEE alarms to traps and map standard SLEE usage parameters to SNMP get/set operations. This is a bit tricky, but in my opinion do-able as it does not introduce any proprietary extension to the container and the sbb developer can continue to use standard jain slee APIs.

For SIP servlets, I am not sure how we will enable the servlet to raise traps or allow get/set operations. Maybe we need to expose a similar API to the servlet.

Just my two cents.

On Jan 19, 2011 3:34 PM, "Jean Deruelle" <jean.d...@gmail.com> wrote:

> I will ask around the office to see if anyone has any additional

> feedback or information we can ...

OK good. upgrading to version 3 will be indeed the first step but I actually just saw the AS Team is moving forward with this See https://issues.jboss.org/browse/JBPAPP-5592

It will also be compatible with AS 5.1 version of JBoss. I will be working with the JBoss AS issue owner on this as we gather feedback to make sure this fits Mobicents needs.

As a side note Jopr on the other side support v3 (and uses SNMP4J under the covers IIRC)

 
>
>
> > Just to make sure I get you right, you want a trap per call failed over or

> > just only ...

That is doable.

 
>
> A trap per call would destroy any monitoring system!

This way people can get to their homes early saying the monitoring system is down :-)

 
>
>
> > I would be interesting to hear your requirements have more details about

> > that, but Jo...

I think all of this is supported by the Jopr Alert + SNMP supports, will double check

 
>
> > Yes the first step scheduled for issuehttp://code.google.com/p/mobicents/issues/detail?id=1...

Persistency will only be achievable on top of JBoss AS 7.

>
>
> > This is the intent but if you could provide a (exhaustive?) list of what you

> > encompass ...

So dynamic registration of SNMP attributes and tables from your servlet ? Through an API (or access to the underlying SNMP4J API) or through an application MIB that would be registered/unregistered by the container automatically on application startup/shutdown ? If you can be more specific on how at usage you would like to register those per servlet SNMP specific attributes it would be nice.

As a side note :This is already possible with the current implementation for JMX (See below) but I guess you would like to remove this JMX layer completely.

(Inside the snmp-agent.sar directory, you'll see a file called attributes.xml. This file associates SNMP OIDs with JMX attributes.
So for each item in your MIB, you need to expose that information via a JMX MBean, and then add the OID->JMX association to attributes.xml.
The agent itself neither knows nor cares about your MIB, so you need to be careful to specify the OIDs in attributes.xml properly.)

>
> I would also like to be able to generate SNMP traps from my servlet.

Ok, will add it as well.

 
>
>
> Ideally, if Mobicents has some SNMP configuration/alarm/event model

> (either independently...

I'm not sure you need JOPR for that, I think the set of metrics can already be fetched

 
>
> , and have the
> ability for servlets to generate their own bespoke metrics for JOPR

> displa...

[aayush]

One more sidenote to the type of Alarms Peter had mentioned-

Alarms need to be of 2 types: ADMC and ADAC.

Automatically Detected and Manually Cleared Alarms (ADMC) where alarms are cleared from the EMS manually.

Automatically Detected and Automatically Cleared (ADAC) Alarms, where alarms are cleared by the container/application. In JSLEE we can raise and clear alarms.

Regards

Aayush

On Jan 23, 2011 11:08 AM, "aayush" <abhatnag...@gmail.com> wrote:

In my opinion, we should first freeze the managed entities for Mobicents -

1. MSS
2. JAIN SLEE
3. Others?

Based on this, we should model standard MIBs for each managed entity. Each Mobicents server would be maintaining metrics and exposing it using MBeans for example.

Hence our MIBs need to tie back closely to these existing MBeans.

The SNMP agent for Mobicents can be deployed as a standalone SAR binary similar to what JBOSS does now and shipped as a common module.

Based on which server is being operated (MSS or SLEE), MIBs can be dropped into a specific folder of the binary and registered with the agent during startup.

These MIBs would behave as "standard mibs" for the mobicents distribution.

The other requirement is for the application (sip servlet or sbb) to raise traps and expose information over SNMP.  MIBs in this case need to be modelled manually and provided to the EMS.

For MIB implementation in JAIN SLEE applications, we can escalate standard SLEE alarms to traps and map standard SLEE usage parameters to SNMP get/set operations. This is a bit tricky, but in my opinion do-able as it does not introduce any proprietary extension to the container and the sbb developer can continue to use standard jain slee APIs.

For SIP servlets, I am not sure how we will enable the servlet to raise traps or allow get/set operations. Maybe we need to expose a similar API to the servlet.

Just my two cents.

>
> On Jan 19, 2011 3:34 PM, "Jean Deruelle" <jean.d...@gmail.com> wrote:
>

> > I will ask around the office to see if anyone has any additional

> feedback or information we can ...

>
> OK good. upgrading to version 3 will be indeed the first step but I actually just saw the AS Te...

>
>
>  
> >
> >
> > > Just to make sure I get you right, you want a trap per call failed over or

> > just only ...

>
> That is doable.
>
>
>  
> >
> > A trap per call would destroy any monitoring system!
>

> This w...

>
>
>  
> >
> >
> > > I would be interesting to hear your requirements have more details about

> > that, but Jo...

>
> I think all of this is supported by the Jopr Alert + SNMP supports, will double check

 
>
> > Yes the first step scheduled for issuehttp://code.google.com/p/mobicents/issues/detail?id=1...

>
> Persistency will only be achievable on top of JBoss AS 7.

>
>
> >
> >
> > > This is the intent but if you could provide a (exhaustive?) list of what you

> > encompass ...

>
> So dynamic registration of SNMP attributes and tables from your servlet ? Through an API (or ac...

>
>
>
> >
> > I would also like to be able to generate SNMP traps from my servlet.
>

> Ok, will add...

>
>
>  
> >
> >
> > Ideally, if Mobicents has some SNMP configuration/alarm/event model

> (either independently...

>
> I'm not sure you need JOPR for that, I think the set of metrics can already be fetched

>
>
>  
> >
> > , and have the
> > ability for servlets to generate their own bespoke metrics for J...

> displa...

>
> This will be achievable by exposing JMX metrics and then writing a Jopr Plugin to fetch them an...

[Jean Deruelle]

Thanks for the valuable feedback Aayush. I was contacted privately wrt this thread as well regarding supporting SNMP AgentX (RFC2741) so that Mobicents can act assubagent as well.

As I was skimming through this RFC, here were my thoughts :

"So Basically JBoss AS (and Mobicents) will be a subagent talking to a Master Agent on the network. This is another configuration we need to support in addition to run standalone for architectures where there is no Master Agent on the network

 

I need to check if the specification support this mode where the subagent can be accessed directly if there is no Master Agent on the network by example.

The AgentX architecture is actually quite interesting in the sense that the concept could be kind of reused for JBoss AS internally (with colocation handling no messages over the network just in JVM API calls)

JBoss AS registering a Master Agent (rather Master Sub Agent) and each part of the container would be a subagent (Mobicents would be a subagent, JBoss Web would be a subagent, JMS, JBoss Cache would be subagents,...)  and register their specific MIB into the JBoss AS master MIB.

Same thing for applications, they could register as a sub agent of a particular component (Mobicents in our case) and register their MIB on application startup."

Creating an API to enable the servlet to raise traps or allow get/set operations would be needed for HTTP Servlets as well as this issue is broader than Mobicents and reach out to JBoss AS itself as well to be more manageable through SNMP in Telco environment. See JBoss AS Issue (that I just discovered lat week) https://issues.jboss.org/browse/JBPAPP-5592

Jean

[Eduardo Martins]

There are probably use cases which can take advantage of JAIN SLEE
Alarms extending its cover to SNMP, but the same can be said for JAIN
SLEE Tracers (for instance WARNING and SEVERE traces). And both should
be configurable, Alarms just on/off, and for Traces a threshold level.

Dunno if we can plug in the SNMP component code to use something, for
instance JMX, to clear the alarm, but that's something as useful as
defining the Tracer level from log4j, a must in my humble opinion.

-- Eduardo
..............................................
http://emmartins.blogspot.com
http://redhat.com/solutions/telco

[Sachin Parnami]

IMHO
Adding Stats as well would be a good idea as well, to track load is being taken and if logged properly can also track the number of failed calls.

Regards,
Sachin Parnami

[Oleg Kulikov]

Monitoring of SLEE traces over SNMP in my opinion is not required. Alarm only - on/off. Application/server decides what switch alarm on: link down, excessive call delay, excessive failure, what ever else. 

In this case SLEE can be pluggable into network trouble ticket system and engineer on duty can catch problem. 

--Oleg

2011/1/24 Eduardo Martins <emma...@gmail.com>

[Eduardo Martins]

Agree.

-- Eduardo
..............................................
http://emmartins.blogspot.com
http://redhat.com/solutions/telco

[Eduardo Martins]

Well, I would say that a Alarm is always a better choice than a SEVERE
level trace, in production, but I don't think it's a big issue to give
the option, probably adds almost nothing in the container's code...

-- Eduardo
..............................................
http://emmartins.blogspot.com
http://redhat.com/solutions/telco

[Vladimir Ralev]

You can also use the SNMP log4j appender. This way you can just log things and selectively convert them to SNMP traps.

The more interesting problem is how to capture the health reliably. Clustering already has low-noise log messages for joining/failing, you can also capture congestion control warnings easily, but deadlocks or stuck calls might be a problem - may be you will have to audit ongoing calls periodically.

[aayush]

Does JBOSS expose its SNMP capabilities as a service? (using jndi or something).

That would make it pretty straightforward for the container to map its internal SLEE alarms to SNMP alarms.

On Jan 25, 2011 12:08 AM, "Vladimir Ralev" <vladimi...@gmail.com> wrote:

You can also use the SNMP log4j appender. This way you can just log things and selectively convert them to SNMP traps.

The more interesting problem is how to capture the health reliably. Clustering already has low-noise log messages for joining/failing, you can also capture congestion control warnings easily, but deadlocks or stuck calls might be a problem - may be you will have to audit ongoing calls periodically.

On Thu, Jan 13, 2011 at 11:10 PM, pdunkley <peter....@crocodile-rcs.com> wrote:
>
> Hello,
>

>...

[Vladimir Ralev]

The appender and log4j is not technically a service, but you can think of it as if it is. It is built in, starts at boot time and can be accessed thought the logger which is available from anywhere in the app server thus capturing events from the low-level places as well as the apps. By default it is commented out in the jboss-log4j.xml but it is easy to enable it and works flawlessly (a small hack is needed sometimes if it doesnt work right out of the box).

   <!-- Log events through SNMP

   <appender name="TRAP_LOG" class="org.apache.log4j.ext.SNMPTrapAppender">

     <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>

     <param name="ImplementationClassName" value="org.apache.log4j.ext.JoeSNMPTrapSender"/>

     <param name="ManagementHost" value="127.0.0.1"/>

     <param name="ManagementHostTrapListenPort" value="162"/>

     <param name="EnterpriseOID" value="1.3.6.1.4.1.24.0"/>

     <param name="LocalIPAddress" value="127.0.0.1"/>

     <param name="LocalTrapSendPort" value="161"/>

     <param name="GenericTrapType" value="6"/>

     <param name="SpecificTrapType" value="12345678"/>

     <param name="CommunityString" value="public"/>

     <param name="ForwardStackTraceWithTrap" value="true"/>

     <param name="Threshold" value="DEBUG"/>

     <param name="ApplicationTrapOID" value="1.3.6.1.4.1.24.12.10.22.64"/>

     <layout class="org.apache.log4j.PatternLayout">

            <param name="ConversionPattern" value="%d,%p,[%t],[%c],%m%n"/>

     </layout>

   </appender>

   -->

There are also some good examples of SNMP events in the jboss-log4j.xml

 <!--

       | Logs these events to SNMP:

           - server starts/stops

           - cluster evolution (node death/startup)

           - When an EJB archive is deployed (and associated verified messages)

           - When an EAR archive is deployed

       

   <category name="org.jboss.system.server.Server">

     <priority value="INFO" />

     <appender-ref ref="TRAP_LOG"/>

   </category>

  

   <category name="org.jboss.ha.framework.interfaces.HAPartition.lifecycle">

     <priority value="INFO" />

     <appender-ref ref="TRAP_LOG"/>

   </category>

   <category name="org.jboss.deployment.MainDeployer">

     <priority value="ERROR" />

     <appender-ref ref="TRAP_LOG"/>

   </category>

   

   <category name="org.jboss.ejb.EJBDeployer">

     <priority value="INFO" />

     <appender-ref ref="TRAP_LOG"/>

   </category>

   

   <category name="org.jboss.deployment.EARDeployer">

     <priority value="INFO" />

     <appender-ref ref="TRAP_LOG"/>

   </category>

   -->

Similar categories can be isolated for a lot of the error cases. IMO the bigger problem is to not only capture the errors but also to capture diagnosting information in production - having a thread dump, a memory snapshot, details about the failed call, how long the app took to process the requests, is there a deadlock in the app or in the container, etc.

[aayush]

I have used this config..but the problem is that the traps would then be bound to the log levels.

Maybe if SNMP could be exposed as a service,then it would be ideal.

> Does JBOSS expos...

[Vladimir Ralev]

Why log levels is bad? It looks like another advantage to me. For example jboss-log4j.xml can be reconfigured instantly at runtime (just by saving it) to adjust and adapt the desired output/noise based on loglevels depending on what is expected from the system at given moment - you can turn off temporary noisy events and so on.

[aayush]

The kind of functions that Jean described - sub agents registering to the master agent etc would become easier to implement if the SNMP capabilities are exposed as a service.

Maybe we can extend the current SNMP capabilities of Jboss and bind them to a unique JNDI name available across the container to all applications.

On Jan 25, 2011 2:38 PM, "aayush" <abhatnag...@gmail.com> wrote:

I have used this config..but the problem is that the traps would then be bound to the log levels.

Maybe if SNMP could be exposed as a service,then it would be ideal.

>
> On Jan 25, 2011 1:53 PM, "Vladimir Ralev" <vladimi...@gmail.com> wrote:
>

> The appender an...

>  <!--
>
>        | Logs these events to SNMP:
>
>            - server starts/stops
>

>            ...

[Jean Deruelle]

Yes both accessible through JMX and JNDI would be ideal

[aayush]

I would definitely like to configure my log levels independent of the Alarm severity.

Also, logs generated by each jboss service at INFO level are escalated as traps,which makes it virtually impossible to model my MIB.

The MIB needs to contain the superset of all possible alarms raised by a network element.

At the EMS..it looks like i am not watching alarms but a server startup log.

If log4j configs provides me with this capability of independent configuration of alarm levels ,then its ok.

Secondly, not all alarms need to be raised on SNMP in the first go. Some informational alarms are resolved locally at the OMC. It is only when a certain alarm is raised frequently and crosses a configured threshold, do we raise it to the EMS with a higher alarm severity.

Eg. An alarm of MINOR severity raised thrice within 1hr needs to be escalated as WARN level trap to the EMS.

Currently with Jboss logging configs, all alarms are raised to the EMS unconditionally and worse all alarms are ADMC type.

On Jan 25, 2011 2:47 PM, "Vladimir Ralev" <vladimi...@gmail.com> wrote:

Why log levels is bad? It looks like another advantage to me. For example jboss-log4j.xml can be reconfigured instantly at runtime (just by saving it) to adjust and adapt the desired output/noise based on loglevels depending on what is expected from the system at given moment - you can turn off temporary noisy events and so on.

On Tue, Jan 25, 2011 at 11:08 AM, aayush <abhatnag...@gmail.com> wrote:
>

> I have used this ...

[Vladimir Ralev]

I think log4j can accomodate all of this without changes in some way. Do you want the SNMP service to maintain the state/counters of the alarms (3 times within an hour) or is this app responsibility?

If you want to count the events may be the best place to keep the counter variable is in the app since it may want to try to recover somehow and you can dedicate a separate category just for SNMP instead of logging levels alone.

public class AppLogic {

 static Logger logger = Logger.getLogger(AppLogic.class);

 static Logger trapLogger = Logger.getLogger("SNMP_SEVERE_LOGGER");

 static int count;

 public void warn() {

  logger.warn("warn");

  if(count%3==0) {

    logger.error("Three times warn");

    trapLogger.warn("Three times warn, emit SNMP");

[Jean Deruelle]

Little update here, I gathered all feedback so far from community and customers in a document located here.

https://docs.google.com/document/d/1sQwY1IOby5sNgwLFAMdgV-vBSFSu_32GjFYelPsWHIM/edit?hl=en

Jean

[Eduardo Martins]

We should do instead article in Mobicents wiki at jboss.org, Google
doc is almost invisible...

-- Eduardo
..............................................
http://emmartins.blogspot.com
http://redhat.com/solutions/telco