Just like GWT generates different JavaScript files for each browser (user agent), gwt-log will cause different JS files for each compile time log level. Combined you will get different JS files for each combination of user agent / compile time log level.
In other words, users running log level WARN or ERROR should not pay the price of any of the logging at lower levels such as debug.
Be sure you are checking the size of the JS being sent to the browser, not the total amount of JS being generated.
Fred