Change Synchronize() Logging level

[ppa...@odesk.com]

Hello,

I use the latest version of Synchronize in order to upload an artifact to an S3 bucket through an exec-maven-plugin. This is part of our continuous integration process, offered by a Bamboo server. Nevertheless, my build agent machine encountered severe heap memory problems due to the very large number of log lines generated by the jets3t module during the synchronization! Especially the debug ones during the upload, are killing us! I failed to find a way to change the logging level using the command line parameters. 

Please, can you help? In fact, why do we still get extensive debug info even when specifying the -q (quiet) parameter? I would expect the debug info to go away...

[James Murty]

Hi,

Synchronize prints information to the console by default, but not if you use the -q (quiet) parameter and in any event these aren't log messages precisely.

JetS3t's low-level logging uses Apache Commons logging [1]. If you are seeing a lot of debug messages I would guess there is a rogue configuration file somewhere that turns on DEBUG logging in general and JetS3t is obeying it, which would definitely produce a huge amount of pointless logs. If you were merely missing logging config files, you would get a brief error message along the lines of the following:

    log4j:WARN No appenders could be found for logger (org.jets3t.service.Jets3tProperties).

    log4j:WARN Please initialize the log4j system properly.

The JetS3t codebase (and zip file download) includes example configuration files `commons-logging.properties` and `log4j.properties` that set up sane/minimal logging settings, dropping these or similar config files into your process' classpath might be the quickest way to solve the problem.

Hope this helps,

James

[1]: http://jets3t.s3.amazonaws.com/toolkit/guide.html#logging

---
jamesmurty.com

--
You received this message because you are subscribed to the Google Groups "JetS3t Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jets3t-users...@googlegroups.com.
To post to this group, send email to jets3t...@googlegroups.com.
Visit this group at http://groups.google.com/group/jets3t-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[ppa...@odesk.com]

Hello,

Well i'm not using any apache commons or log4j configuration files anywhere in my project and i doubt if any 3rd party dependencies referenced through the public Maven repository contain their own. And i really don't want to introduce a new and confusing configuration file just to solve this problem. 

Can't we think of some hack? 

[ppa...@odesk.com]

I removed all my commons-logging dependencies and log4j dependencies from my project and still nothing changes. I searched into my final, fat jar for rogue configuration files but could not find anything. I though maybe this is a Maven issue and used the Maven -q switch but nothing changed.

[ppa...@odesk.com]

I used the configuration files you mentioned with no results. I put them in the folder where the fat gar resided, inside the jar, in the same folder as the POM, in the Maven executable folder and nothing changed. I think the problem has to do with the use of the exec-maven-plugin to run the synchronization....

[James Murty]

Can you be more precise about what debug messages are being output by JetS3t? I want to be sure the log messages you're talking about are actually logging system messages, as opposed to console feedback text.

I'm not too familiar with Maven. To be honest, I know it just well enough to avoid it whenever possible.

The thing to look out for, though, isn't commons logging or log4j dependencies but logging configuration files or settings. If there was no logging config then very little, if anything, would get logged by the commons logging system so there must be something somewhere. For the config files included with JetS3t to be applied they will need to be in the exec-maven-plugin process classpath, wherever that is.

As far has hacks go, a couple of less-than-ideal options I can think of:

- Explicitly disable commons logging using a -D switch in your Maven config, e.g. -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.NoOpLog

- Redirect all stderr output to /dev/null. This assumes the log messages are actually going to stderr, which depend on the logging configuration, so maybe a catch 22.

---
jamesmurty.com

--

[ppa...@odesk.com]

These are the ones that bloat everything. I get a 80MB log file for 30MBs of files to upload.

13:29:57.162 [Thread-3] DEBUG org.apache.http.wire - >> "it[0x9f][0x89]1&0[0x1f]>R.qe[0x1][0xe2][0x9c]a [0xc8]0[II[0x1a][0x89]a[0xe4]$;;[0x16][0xff][0xba][0xfa]x[0x1][0x89]:[0xd1][0xd2][0x91].[0x15][0xe]([0x10][0xcc]0E[0x15]A"[0x86][0xf2][0xcc][0xff]F[0x9a]N[0xe4][0x98][0x95][0xff][0xab][0xc2][0xc8][0xe5][0x84]}R6Z[0x87][0xec]`([0x15]![0x99][0xfb]>[0xad][0x6][0xad][0xc]"[0x89]K[0xc5][0x14],+[0x9a][0x0]YI[0x9b][0xa0][0x2][0x92][0x1f](/[0x97]!fH[0x1b]5[0xef][0x80][0xe2]}"[0x90]2[0x1a]I<'V[0xfc][0xed]o[0x9c][0xe6][0xd3][0x5][0xe4]QH[0x8b][0xad]r[0x96]n[0xe5][0x80][0x81]b[0xda][0xe1][0xb2][0xbd][0xbe]#Q[0xc0]zP[0xa8],V5[0xf8][0xad])[0x9a]x[0x3][0x96]3,![0xa0]@[0xc8][0x8][0x12][[0xbe][0x9a][0xeb][0x11]{y[0xbe][0xb2][0xa0][0x18] [0xdc][0x97]#[0xc4]'[0xb0][0xfe]q[0xa2]|[0xd0][0x1b]i[0x9b]x[0xb9][0x93] <[0xc9]E[0xce][0xc2][0x8a][0x17][0xbf][0x9d][0xb9][0xc6]#[0xa5];[0x10]B[0xfc][0x13][0xc4][0xfd][0x6][0x8c][0xa3][0xb0]&=[0x19][0x81] [0x6]J$[0xe0][0xec]PWG[0xfe]$b[0xb3][0x95][0xa9]o6[0xa3][0x11][0xe5][0xc5][0xde]8[0x91][0x9f][0x18]=[\n]"

build 21-May-2013 13:29:57 13:29:57.164 [Thread-3] DEBUG org.apache.http.wire - >> "[0xc2][0xd6];x[[0x8][0x1f]e[0xf1][0xfc][0xb5]q8[0x88][0xee]^[0xc]5[0x17][0xd1][0xd4]B[0xa9][0xb4]D?[0x94]Q[0x80][0xe1][0x9][0xd4][0x1f]([0xa9][0xe3]V[0xca][0xae][0x18][0xc6][0x4][0x8][0xdd]:[0xf1]|P?[0x86][0xa6]F.OS[0x85][0xd2][0xef][0xa8]n[0x8][0xac][0xd2][0xe9]^[0xbb]G/n[0xba][0x83]N[0xeb][0xb6][0x95]c[\r]0[0x96][0xa5]C[0xfc][0x4][0xfe][0x7]\[0xbf][0x89][0xa2]7[0xd3][0x81]E[0x1a][0xb2]{[0x9][0x14]C[0xb8][0xbf][0xa2](X[0xbc][0xbb]_r[0xd0]+[\n]"

build 21-May-2013 13:29:57 13:29:57.164 [Thread-3] DEBUG org.apache.http.wire - >> "[0x17][0xe4]^[0xe4]p[0xe8][0xe7]$z[0x1f]8[0xbb][0x3]O[0x1c]@[0xa8][0x1f][0x0],[0xe3][0x88]#[0x8][0x8f][0xe1][0x18][0x1c]xH[0x86][0x8][0xfa][0xf0]D[0x1a][0x82][0xcd][0x7]@T[0xc][0x9e]f[0xe2][0xed][0xb0][0xd8][0x8a][0xd2][0xfd][0x13][0xc][0x9e]t[0xc2]1[0x1a]B7[0xd6][0xa8][\n]"

build 21-May-2013 13:29:57 13:29:57.164 [Thread-3] DEBUG org.apache.http.wire - >> "*l[0x8c][0xa6]X[0xb9][0x82][0xee][0xa7]a[0x0][0xea][0x9b][0x18][0xfd][0xcb]XA[0xcd][0xea]aE[0xe9],[0xb9][0x6][0xe3][0x0][0x19]C[0x98][0x1b]![0xe4] [0xc4][0xbd][0xe2]na[0x9c][0xe][0xe0][0x2]P[0xe1][0xb1]IP-[0x11]B[0xa4][0xf2][0x90]W[0xca]*[0x99][0xf9]T[0xb2]S`[0xf2][\n]"

build 21-May-2013 13:29:57 13:29:57.164 [Thread-3] DEBUG org.apache.http.wire - >> "[0xd0][0xec][0xc5][0xb][0x11]OF2[0xa1][0x10]m[0x92][0xd1][0xc][0xfd][0x1b][0xf1][0xd2][0x86][0x12]`[0xb0]4d&[0x85][0x8][0x97]y[0xf8][0x8b][0x19][0x92][0x5]|[0xbb]U[0x83][0xab][0xea][0x8e][0xcd][0x8d][0x9f][0xa7][0xd9]SH[0xa8][0xb2][0x9c][0xba]![0xc7][0xa2][0x8b][0x1a]UWV[0xe3][0x11]'[0xf6][0x17][0xa7]s[0xe9]ORy;[0x2][0x13][0x85][0x1d]}[0xf][0xbc][0xf4][0x81]-[0x9f]U[0xea]E5[0x1b][0x8c][0xdd][0xa1]V[0xfa]*[0x8a][0xd4][0xc][0x95]G[0x89][0xf7][0xe]L[0xe6][0x1b][0xca]$F[0xc1][0xdf][0x81][0xad][0x8d]0a[0xb]:[0xc8]Q6[0x87][0xc8][0x83]n[0xdb][0xa7][0xdd][0xa0]c[0x90]X[0xba][0xc0][0xcc]X[0xc9]t2[0xa8]'.[0xdb][0xf4];*[0x0][0xa8][0x3][0x88][0xbe][0xbe][0x4]:[0x0][0xf0]U[\n]"

build 21-May-2013 13:29:57 13:29:57.165 [Thread-3] DEBUG org.apache.http.wire - >> "[0xe6][0xa5])[0x87][0xfe]AG[0x92][0xb3][0xf8][0xdc][0xca][0xfd]%6[0x8c][0xe5]/[0xc9][0xdf][0xc2][0xf4]M. bH?[0x1c]y[0x91][0xd1][0x5][0x85][0xc8][0xd2][0x13]r[0xd8]kXz[0x3][0xe];[0xc0]|[0xcc][0xda]7[0xd1][0xc7][0xc9]_w[0xc8][0x8d][0xae][0x9b]e[0xd3]m[0xde][0x14](H[0xb6][0xa5][0xf0][0xf9][0xd4][0xb0]k[0xb8][0xae][0x1]K[0x96][0xdd]YJh[0x1b]H[0xc2][0x1c]\[0xa9]l[0xbb][0x98][0x8c][0xee]d[0xc2][0xa0][0xba]K[0xde]Z[0xe5][0xd3][0xec]<[0x7][0x87]q[0xfb][0x91]Rk7[0xd6][0xa9][0xae]>[0xbe][0x15]i=&r[0xf8][0x9d]6[0x91][0xf5]z'K[0xba][0x11]][0xaa][0x8e](i[0x11][0x8e][0x14][0xbd][0xe5]2;[0xb9][0x1c][0xe7]*[0xa1][0x7]' [0x94][0xb4]y[0xf3]:[0x13]]y[0xf1][0xdf]E/[0x13][0xe2]2[0x94]^[0xce]t[0xc3][0x0][0xb4][0xf9]l[0xa2]e[0x94]$^[0xde][0xdf]-s[0xce][0x12][0xac][0x81][0x3][0xbf][0xa0][0x88][0xb7][0x4][0x7][0x9f][0xeb][0xed][0xae]-[0x0][0x11][0xa4]a[0x81][0x9d][0x8b]BM[0xd5][0xc4][0xc9]L{[0x1a]J[0xb9][0xf4]B[0xa5][0x17][0x93](b[0xe7][[0x6][0x9][0x94][0xb][0xeb][0xbf][0xba][0x1d]X[0xdf]l[0xb2][0xe][0x13][0xea]&[0xeb][0xf0][0xfc]1^^[0x8]a[0xaf]n[0xad]?I[0x12][0x8][0xc6][0x9f][0xc3][0xcd]@[0x87][0xda][0xd4]Co1^[0xca][0xe4][0xe3][0x18][0xce]o[0x1b][0xec][[0xe][0xfc][0x9b]na[0xeb][0xdb]`u[0xc9]Z7Y[0xcf][0xce]B[0xeb][0xe1][0x4][0x19]yc-[0x83][0xcc]\[0xb6][0x5]p:[0xf7][0xe5][0x98]:Y[0x9f][0x87][0x89][0x9a]m$[0x11]{[0xff]{/[0x8c]&[0xc9]FR)o[0x8b]"<o[0xee][0xb8][0xfd]Rbc[0xe][0xa8] ,[0x80][0xab][0x89][0x8f][0xd5][0xe6][0xb6][0xdb]/[0xa5][0x1e][0x83][0xc8][0xa4]~[0xf][0xe5][0xd8][0x9][0xf6][0x8b][0xb7][0x85]p%G[0xde]x[0x8]u5[0xb3][0x0][0xa7][0xf5]m[0x1][0xc0][0xe6][0xd4]X[0xf4]#v?[0x8e]o[0xb0][0x11][0xe9][0x5][0x9f][0x94][0x8a][0x1e]K6z[0xab][0x9a]ld[0xe7][0xc6][0xbf]OTl~[0xe3]j[\n]" 

By explicitly disabling commons logging using the -D switch in the command line, did the trick! i just hope i'm not missing anything really interesting. I guess that i will fail to see any errors...

I insist that i searched everything and didn't find any configuration files. I removed all logging and log4j dependencies in order to be sure that even if there were any rogue configuration files, the system will not be able to load or process them! I removed the dependencies even from the jets3t libraries referenced through Maven! Without these references, the logging system should not be able to work...

[James Murty]

I'm glad that -D hack helped. 

You do have a basic problem remaining though with your logging setup. Those DEBUG messages aren't actually generated by JetS3t, they're from the lower-level HttpClient library JetS3t uses.

So it definitely looks like you somehow had all debug logging turned on for everything, not what you want at all. 

--
---
jamesmurty.com

[ppa...@odesk.com]

Well, i can assure you, this is not the case! Running my application, everything works as expected, log-wise. Only when initiating the Synchronize through exec-maven-plugin, i get all this stuff! Somehow, the DEBUG level is enforced for everything. Even for Jets3t, as other logs encountered are:

21-May-2013 13:29:55 13:29:55.479 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.j.s.i.r.h.RestStorageService - Response entity: org.apache.http.conn.BasicManagedEntity@2441db59

build 21-May-2013 13:29:55 13:29:55.479 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.j.s.i.r.h.RestStorageService - Entity length: 0

build 21-May-2013 13:29:55 13:29:55.479 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.j.s.i.r.h.RestStorageService - Received expected response code: true

build 21-May-2013 13:29:55 13:29:55.479 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.j.s.i.r.h.RestStorageService -   expected code(s): [200, 204].

build 21-May-2013 13:29:55 13:29:55.479 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.j.s.u.RestUtils$ThreadSafeConnManager - Released connection is reusable.

build 21-May-2013 13:29:55 13:29:55.480 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.a.h.i.conn.tsccm.ConnPoolByRoute - Releasing connection [HttpRoute[{s}->https://odesk-agora-prod.s3.amazonaws.com:443]][null]

build 21-May-2013 13:29:55 13:29:55.480 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.a.h.i.conn.tsccm.ConnPoolByRoute - Pooling connection [HttpRoute[{s}->https://odesk-agora-prod.s3.amazonaws.com:443]][null]; keep alive indefinitely

build 21-May-2013 13:29:55 13:29:55.480 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.a.h.i.conn.tsccm.ConnPoolByRoute - Notifying no-one, there are no waiting threads

build 21-May-2013 13:29:55 13:29:55.481 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Cleaning up REST metadata items

build 21-May-2013 13:29:55 13:29:55.481 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Removed header prefix x-amz- from key: x-amz-request-id=>request-id

build 21-May-2013 13:29:55 13:29:55.481 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Leaving header item unchanged: Date=Tue, 21 May 2013 13:29:56 GMT

build 21-May-2013 13:29:55 13:29:55.482 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Parsing date string 'Tue, 21 May 2013 13:29:56 GMT' into Date object for key: Date

build 21-May-2013 13:29:55 13:29:55.483 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Leaving HTTP header item unchanged: Content-Length=0

build 21-May-2013 13:29:55 13:29:55.483 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Removed header prefix x-amz- from key: x-amz-id-2=>id-2

build 21-May-2013 13:29:55 13:29:55.483 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Ignoring metadata item: Location=/odesk-agora-prod

build 21-May-2013 13:29:55 13:29:55.483 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Leaving HTTP header item unchanged: Content-Type=application/octet-stream

build 21-May-2013 13:29:55 13:29:55.483 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.ServiceUtils - Ignoring metadata item: Server=AmazonS3

build 21-May-2013 13:29:55 13:29:55.484 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - uploads.storeEmptyDirectories=true

build 21-May-2013 13:29:55 13:29:55.484 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - filecomparer.skip-upload-of-md5-files=false

build 21-May-2013 13:29:55 13:29:55.484 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - filecomparer.skip-symlinks=false

build 21-May-2013 13:29:55 13:29:55.535 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - upload.ignoreMissingPaths=false

build 21-May-2013 13:29:55 13:29:55.535 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - upload.max-part-size=1073741824

build 21-May-2013 13:29:55 13:29:55.538 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - filecomparer.bucket-listing.odesk-agora-prod=null

build 21-May-2013 13:29:55 13:29:55.547 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - threaded-service.admin-max-thread-count=20

build 21-May-2013 13:29:55 13:29:55.547 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - threaded-service.max-thread-count=2

build 21-May-2013 13:29:55 13:29:55.547 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - httpclient.max-connections=20

build 21-May-2013 13:29:55 13:29:55.548 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.jets3t.service.utils.FileComparer - Listing objects in 'odesk-agora-prod' using 1 prefixes: [agora/hystrix-dashboard]

build 21-May-2013 13:29:55 13:29:55.552 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - threaded-service.admin-max-thread-count=20

build 21-May-2013 13:29:55 13:29:55.552 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG org.jets3t.service.Jets3tProperties - threaded-service.ignore-exceptions-in-multi=false

build 21-May-2013 13:29:55 13:29:55.552 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.j.s.m.ThreadedStorageService$ThreadGroupManager - Started ThreadManager

build 21-May-2013 13:29:55 13:29:55.553 [org.jets3t.apps.synchronize.Synchronize.main()] DEBUG o.j.s.m.ThreadedStorageService$ThreadGroupManager - Thread 1 of 1 has started

build 21-May-2013 13:29:55 13:29:55.554 [Thread-2] DEBUG org.jets3t.service.Jets3tProperties - s3service.disable-dns-buckets=false

build 21-May-2013 13:29:55 13:29:55.555 [Thread-2] DEBUG org.jets3t.service.Jets3tProperties - s3service.s3-endpoint=s3.amazonaws.com

build 21-May-2013 13:29:55 13:29:55.555 [Thread-2] DEBUG org.jets3t.service.Jets3tProperties - s3service.s3-endpoint-virtual-path=

build 21-May-2013 13:29:55 13:29:55.555 [Thread-2] DEBUG org.jets3t.service.Jets3tProperties - s3service.s3-endpoint-https-port=443

To unsubscribe from this group and stop receiving emails from it, send an email to jets3t-users+unsubscribe@googlegroups.com.

To post to this group, send email to jets3t...@googlegroups.com.
Visit this group at http://groups.google.com/group/jets3t-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
---
jamesmurty.com