Jenkins and GNU/Linux OS Builds

[Christian Bryant]

I'm in the process of moving my Linux From Scratch build system to be managed by Jenkins (I can check up on the build using a Jenkins Android app).  However, I'm stumbling through examples of non-Java build configuration scripts that some folks have put out there for GCC and Linux kernel builds.  Anyone in the group built a build management framework for a GNU/Linux OS like Linux From Scratch?  From the Jenkins kernel build code below (not mine), I can see that I'm going to have a large number of shell tasks since I'm going to be building (on top of LFS 7.1) a large number (~200) of packages in addition to the kernel, and squirting out an ISO from this (for a Live DVD).  Appreciate any pointers from the local SMEs.  Cheers.

=== Sample Jenkins XML for a kernel build (note use of NullSCM) ===

1. <?xml version='1.0' encoding='UTF-8'?>
2. <project>
3.   <actions/>
4.   <description></description>
5.   <keepDependencies>false</keepDependencies>
6.   <properties/>
7.   <scm class="hudson.scm.NullSCM"/>
8.   <canRoam>true</canRoam>
9.   <disabled>false</disabled>
10.   <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
11.   <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
12.   <triggers class="vector">
13.     <hudson.triggers.TimerTrigger>
14.       <spec>@weekly</spec>
15.     </hudson.triggers.TimerTrigger>
16.   </triggers>
17.   <concurrentBuild>false</concurrentBuild>
18.   <builders>
19.     <hudson.tasks.Shell>
20.       <command># set up build env
21. rm -rf $WORKSPACE/*</command>
22.     </hudson.tasks.Shell>
23.     <hudson.tasks.Shell>
24.       <command># determine latest grsecurity version
25. VERSION=`curl --silent https://grsecurity.net/test.php | egrep -o &quot;test/grsecurity-[0-9]+\.[0-9]+-3\.[0-9]+\.[0-9]+-[0-9]+\.patch&quot; | head -n 1 | egrep -o &quot;3\.[0-9]+\.[0-9]+&quot;`
26.  
27. # grab tarball
28. curl --silent http://www.kernel.org/pub/linux/kernel/v3.0/linux-$VERSION.tar.bz2 &gt; linux.tar.bz2
29. tar -xf linux.tar.bz2
30.  
31. # set it up
32. mv linux-* stage</command>
33.     </hudson.tasks.Shell>
34.     <hudson.tasks.Shell>
35.       <command># download grsecurity
36. PATCH=`curl --silent https://grsecurity.net/test.php | egrep -o &quot;test/grsecurity-[0-9]+\.[0-9]+-3\.[0-9]+\.[0-9]+-[0-9]+\.patch&quot; | head -n 1`
37. curl --silent https://grsecurity.net/$PATCH &gt; $WORKSPACE/stage/grsecurity.patch
38. </command>
39.     </hudson.tasks.Shell>
40.     <hudson.tasks.Shell>
41.       <command># neoice.net local patch
42. cp /usr/src/kernel/neoice.patch $WORKSPACE/stage</command>
43.     </hudson.tasks.Shell>
44.     <hudson.tasks.Shell>
45.       <command># apply patches
46. cd $WORKSPACE/stage
47. patch -p1 &lt; grsecurity.patch
48. patch -p1 &lt; neoice.patch</command>
49.     </hudson.tasks.Shell>
50.     <hudson.tasks.Shell>
51.       <command># kernel config
52. cd $WORKSPACE/stage
53. cp /usr/src/kernel/config $WORKSPACE/stage/.config
54. make oldconfig</command>
55.     </hudson.tasks.Shell>
56.     <hudson.tasks.Shell>
57.       <command>cd $WORKSPACE/stage
58. make -j7 deb-pkg</command>
59.     </hudson.tasks.Shell>
60.   </builders>
61.   <publishers>
62.     <hudson.tasks.ArtifactArchiver>
63.       <artifacts>*.deb</artifacts>
64.       <latestOnly>false</latestOnly>
65.     </hudson.tasks.ArtifactArchiver>
66.   </publishers>
67.   <buildWrappers/>
68. </project>

[Slide]

I use Jenkins to build embedded software (C based) using makefiles and
the like. Building the kernel shouldn't really be any different. What
exactly are you looking for to enhance your current setup?

slide

--
Website: http://earl-of-code.com

[Les Mikesell]

On Thu, May 17, 2012 at 5:41 PM, Christian Bryant
<christia...@ucla.edu> wrote:
> I'm in the process of moving my Linux From Scratch build system to be
> managed by Jenkins (I can check up on the build using a Jenkins Android
> app).  However, I'm stumbling through examples of non-Java build
> configuration scripts that some folks have put out there for GCC and Linux
> kernel builds.  Anyone in the group built a build management framework for a
> GNU/Linux OS like Linux From Scratch?  From the Jenkins kernel build code
> below (not mine), I can see that I'm going to have a large number of shell
> tasks since I'm going to be building (on top of LFS 7.1) a large number
> (~200) of packages in addition to the kernel, and squirting out an ISO from
> this (for a Live DVD).  Appreciate any pointers from the local SMEs.
> Cheers.

Jenkins can run whatever build script you were using before. Do you
want to break out each package as a separate job so they can build in
parallel or as changes show up? Is the source in some source control
system that jenkins can poll?

--
Les Mikesell
lesmi...@gmail.com

[Christian Bryant]

Not sure if you're familiar with LFS (http://www.linuxfromscratch.org/) but there are a variety of ways to build the OS.  I chose jhalfs (http://www.linuxfromscratch.org/alfs/) which "reaches" into the LFS book source XML files to generate make files.  It does, however, require quite a bit of hand-holding, editing and trial-error.  It would make sense to build each package, including the kernel, as a separate job, with a final job representing the final integration of the temporary system with the app packages, then applying the bootscripts.  The final result ultimately should be an ISO of a Live system, so perhaps that is the final job...  

With that as the current setup, I'd like to use Jenkins for a couple different things, the primary "thing" being an automated build management framework that I can monitor remotely and distribute, if so desired.  The secondary "thing" would be to make use of more features and plugins offered by Jenkins that seemingly are only used by Java apps.  Since I'm just starting the design phase of this project, looking at how others have setup their GCC/C-based Jenkins will be helpful.  The OpenSUSE instance of Jenkins may be a good starting point:  http://ci.opensuse.org/

Cheers. 

- CB

[Les Mikesell]

On Fri, May 18, 2012 at 1:04 AM, Christian Bryant
<christia...@ucla.edu> wrote:
> Not sure if you're familiar with LFS (http://www.linuxfromscratch.org/) but
> there are a variety of ways to build the OS.  I chose jhalfs
> (http://www.linuxfromscratch.org/alfs/) which "reaches" into the LFS book
> source XML files to generate make files.  It does, however, require quite a
> bit of hand-holding, editing and trial-error.  It would make sense to build
> each package, including the kernel, as a separate job, with a final job
> representing the final integration of the temporary system with the app
> packages, then applying the bootscripts.  The final result ultimately should
> be an ISO of a Live system, so perhaps that is the final job...

I don't know enough about it to offer specific advice, but basically
you want to think about what should trigger a build. Does each
package update independently somewhere that jenkins can check? Then
have separate jobs for each action that can proceed independently so
you have the option of distributing them. The build results either
have to be archived on the master or the build scripts have to publish
them somewhere that the iso building step can find. You'll need some
versioning concept for this output to allow building updates to happen
currently with generating the last complete iso.

--
Les Mikesell
lesmi...@gmail.com

[Christian Bryant]

I think I have a pretty solid definition for my architecture now, and based on the shell task examples I've seen out there, including the one I copied in this thread, I think now it will be pretty straightforward from the Jenkins CI side of things.

An interesting blog post I came across confirms my suspicion that, whether GCC or G++, it's all pretty much the same and the real work is in the architecture of the build structure, then tying that into the ISO compile.

http://linuxtortures.blogspot.com/2012/04/continuous-integration-with-jenkins.html

Still, if anyone has some great examples of deep-dive shell tasks that really use Jenkins CI as far as possible with GNU Autotools and GCC, I'd love to see them.

Thanks.

Cheers.

- CB