Disable scalatest logging statements when running tests from maven

[Stephen Boesch]

I am repeating here a thread I posted to StackOverflow several weeks ago (and no response) on the same title.  Here is the content.

http://stackoverflow.com/questions/27220196/disable-scalatest-logging-statements-when-running-tests-from-maven

What is the method to disable logging on the scalatest log4j messages:

The log4j.properties is as follows:

    log4j.rootLogger=INFO,CA,FA

    

    #Console Appender

    log4j.appender.CA=org.apache.log4j.ConsoleAppender

    log4j.appender.CA.layout=org.apache.log4j.PatternLayout

    log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n

    log4j.appender.CA.Threshold = INFO

    

    

    #File Appender

    log4j.appender.FA=org.apache.log4j.FileAppender

    log4j.appender.FA.append=false

    log4j.appender.FA.file=target/unit-tests.log

    log4j.appender.FA.layout=org.apache.log4j.PatternLayout

    log4j.appender.FA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c{1}: %m%n

    log4j.appender.FA.Threshold = INFO

    

    ..

    log4j.logger.org.scalatest=WARN

However we are seeing INFO level scalatest log4j messages:

    2014-11-30 14:25:57,263 INFO  [ScalaTest-run-running-DiscoverySuite] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(840)) - hadoop.native.lib is deprecated. Instead, use io.native.lib.available

    2014-11-30 14:25:57,493 INFO  [ScalaTest-run-running-DiscoverySuite] hbase.HBaseCommonTestingUtility (HBaseTestingUtility.java:startMiniCluster(840)) - Starting up minicluster with 1 master(s) and 2 regionserver(s) and 2 datanode(s)

    2014-11-30 14:25:57,499 INFO  [ScalaTest-run-running-DiscoverySuite] hbase.HBaseCommonTestingUtility (HBaseTestingUtility.java:setupClusterTestDir(390)) - Created new mini-cluster data directory: /shared/hwspark/target/

[Stephen Boesch]

I have changed the logging level to WARN across all log4j.properties; however, the logging statements are still being displayed by scalatest:

Some of the messages are of the following format with [ScalaTest ..] in front:

2014-12-26 16:18:03,821 INFO  [ScalaTest-run-running-JoinsSuite] scheduler.DAGScheduler (Logging.scala:logInfo(59)) - Job 3 finished: collect at SparkPlan.scala:84, took 1.638048 s

2014-12-26 16:18:03,852 INFO  [ScalaTest-run-running-DiscoverySuite] hbase.JoinsSuite (HBaseIntegrationTestBase.scala:afterAll(161)) - Test org.apache.spark.sql.hbase.JoinsSuite completed at Fri Dec 26 16:18:03 PST 2014 duration=79

Test org.apache.spark.sql.hbase.JoinsSuite completed at Fri Dec 26 16:18:03 PST 2014 duration=79

Other messages do not have the ScalaTest in front - but in any case the ScalaTest framework is not respecting the log4j.properties category log levels:

2014-12-26 16:18:05,619 INFO  [IPC Server handler 3 on 54689] FSNamesystem.audit (FSNamesystem.java:logAuditEvent(6725)) - allowed=true ugi=steve.hfs.1 (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/steve/hbase/data/default/metadata/e76dd99e71e4deb8d272d61103df3843/.tmp/8e30b5e3531545a1a6f873e957311fe8 dst=null perm=null

2014-12-26 16:18:05,620 INFO  [DataXceiver for client DFSClient_hb_rs_192.168.253.254,54720,1419639461579_-1940032825_183 at /127.0.0.1:54849 [Sending block BP-1408472754-192.168.253.254-1419639430641:blk_1073741848_1024]] DataNode.clienttrace (BlockSender.java:sendBlock(732)) - src: /127.0.0.1:54703, dest: /127.0.0.1:54849, bytes: 1110, op: HDFS_READ, cliID: DFSClient_hb_rs_192.168.253.254,54720,1419639461579_-1940032825_183, offset: 53248, srvID: DS-458580670-192.168.253.254-54703-1419639459969, blockid: BP-1408472754-192.168.253.254-1419639430641:blk_1073741848_1024, duration: 152000

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalate...@googlegroups.com
To unsubscribe from this group, send email to
scalatest-use...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to a topic in the Google Groups "scalatest-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scalatest-users/IwNy6IuI8-0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scalatest-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bill Venners]

Hi Stephen,

Sorry for the delay in responding. I have been busy with holiday visitors this week. Those log messages are not actually being printed by ScalaTest, but by something you are using from your ScalaTest tests. The reason "ScalaTest" shows up in them is that ScalaTest does change the name of threads when suites and tests are executed, so that if someone has a suite that hangs forever and does a thread dump to investigate, it is more obvious what test and suite is causing the run to hang. Log4J seems to print out the thread name in square brackets, so that can give you a hint as to where these log messages are coming from.

Bill

[Stephen Boesch]

Thanks Bill.

I should have updated that yes the log4j.properties is being pulled from one of the third party jar files when running spark. It is as you describe unrelated to scalatest.  I need to find out how to properly specify the log4j.properties file when running maven - instead of having log4j just pick up whichever one it chooses from somewhere in the classpath.