| The fact that output is stored in two files—log, as with traditional job types, as well as 123.log per step—is a longstanding problem, to be corrected in JENKINS-38381. So that accounts for a factor of two. (JENKINS-38381 also offers the capability for the single copy of the log file to be stored externally. You may use a bunch of disk space on some log server, but $JENKINS_HOME is unaffected.) Then there are console notes, also used in traditional job types but used more widely in Pipeline builds, and consisting of Base64-encoded serialized data embedded in an ANSI escape sequence. These do tend to consume more space than the plain text of the line they appear on (and got bigger after a security fix, alas). However these should only be printed on a tiny fraction of the lines in a typical large build. For example, if you run a sh step, that [Pipeline] sh line will be annotated, but the potentially thousands of lines of output from the process should not be. So typically the contribution to log file size would be minor. Since you claim to be seeing a far larger bloat (in each copy of the log file), and it sounds like you are seeing annotations on every line, something is going wrong. My guess is that you are using the timestamper block-scoped step, which in Pipeline mode does annotate every interior line. (For traditional jobs, unless overridden by a system property, it saves timestamp information to a side file. Also potentially big, though probably not as big.) Until we can think of a way to deal with that, for example by adjusting the Timestamper plugin to skip annotations on most lines if they are being spat out in rapid succession (few users care about millisecond precision in the middle of a 100kLOC log file!), you should try turning this wrapper off and see if that helps. (And of course if we do have an external log service integration with JENKINS-38381, the service may do its own timestamping anyway.) |
