Handling testcases involving reboot during execution of Job on Slave

[Ankit Singhal]

Hi

I need some help in solving the reboot problem.

The Workflow is as follows:

1) Project 1 creates a Windows Installable package.

2) Project 2 is called once the Project1 is completed and Package is pushed to Slave associated with Project 2.

3) Project 2 now runs some commands on Slave 2

Command 1 : Install Package on Slave 2

Command 2 : Run some commands to enable the software.

Command 3 : Reboot the machine.

4) This might break the Job running in Project 2.

5) After reboot , we need to run some more commands on Slave 2 to run some more tests.

6) After tests are completed , the full Job in Project 2 is done.

In nutshell , how to handle the reboot scenario , so that the job is not broken in between.

Regards

Ankit

[Mark Waite]

When I've needed to run something on a freshly booted machine, particularly as part of a series of jobs, I've generally been able to do that by partitioning the work into multiple jobs, with the job which must execute on the newly booted slave being configured to run on the slave.

A single job can't run successfully on the slave node being rebooted (during the reboot), because, as you note, it will fail the job.  Multiple jobs allow you to have the job wait for the slave agent to become available, then it immediately starts the job.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/d2f0f06b-5dcf-4875-92b0-30f0c8dbbe04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ankit Singhal]

Hi Mark,

Thanks for the reply. Can you please help me in detailing the steps of doing it. I tried following

• Created a build-flow name " Start test"
  
• After that created 3 freestyle projects.
  
• Project1 : Before Reboot
  
• Command: echo "Before Reboot"
  
• Project 2 : Reboot
  
• Command : shutdown /r /t 0
  
• Project 3  : After reboot
  
• Command : echo "after reboot"
  
• Then run the above 3 projects from build-flow
  
• The "Start test" project fails after project 2 runs as the node goes offline.
  

 Is there anyway to wait project 3 , till the node comes back online.

Regards

Ankit

[Mark Waite]

My apologies.  My description was wrong.  It makes it sound simpler than it actually is, or at least it sounds simpler than the technique I'm actually using to reboot a slave agent.

I reviewed the Jenkins job which I have that reboots a Windows machine in hopes of reducing the number of times a job will fail due to "busy files".  In that job, I have an ant script which schedules a reboot soon after the current time.  That allows the job which schedules the reboot to complete successfully.

It also requires that a subsequent job be configured to delay its start long enough for the computer to reboot and reconnect the Jenkins slave.  I suspect you'll need to make guesses and watch behavior to decide the length of that delay.

The build.xml file for the reboot task contains this:

  <!-- StackOverflow special thanks for this technique -->

  <scriptdef name="generateguid" language="javascript">

    <attribute name="property" />

    <![CDATA[project.setProperty( attributes.get( "property" ), java.util.UUID.randomUUID() );]]>

  </scriptdef>

  <target name="reboot" description="Reboot this computer">

    <tstamp>

      <format property="current.time" pattern="HH:mm"/>

    </tstamp>

    <tstamp>

      <!-- Schedule reboot 1 minute from now -->

      <format property="reboot.time" pattern="HH:mm" offset="2" unit="minute"/>

    </tstamp>

    <echo>It is ${current.time}, rebooting at ${reboot.time}</echo>

    <generateguid property="guid" />

    <exec osfamily="windows" executable="schtasks.exe" failonerror="true">

      <arg value="/Create"/>

      <arg value="/RU"/> <arg value="Reboot"/> <!-- user name of an Administrator user -->

      <arg value="/RP"/> <arg value=""/> <!-- insert the password of your "Reboot" user -->

      <arg value="/SC"/> <arg value="Once"/>

      <arg value="/ST"/> <arg value="${reboot.time}"/>

      <arg value="/TN"/> <arg value="Reboot-${guid}"/>

      <arg value="/TR"/> <arg value="shutdown /r /t 2"/>

    </exec>

  </target>

Mark Waite

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/d5548546-f61d-4476-981c-245669283299%40googlegroups.com.

[Jason Pyeron]

> -----Original Message-----
> From: Mark Waite
> Sent: Friday, April 24, 2015 8:13
>
> My apologies. My description was wrong. It makes it sound
> simpler than it actually is, or at least it sounds simpler
> than the technique I'm actually using to reboot a slave agent.
>
>
> I reviewed the Jenkins job which I have that reboots a
> Windows machine in hopes of reducing the number of times a
> job will fail due to "busy files". In that job, I have an
> ant script which schedules a reboot soon after the current
> time. That allows the job which schedules the reboot to
> complete successfully.
>
> It also requires that a subsequent job be configured to delay

Many VM (I am assuming the machine is a VM today) system support screen shots. We take screen shots at regular intervals during the reboot process. We then take the md5sum of the bitmap (much easier than "parsing" the image content), once the expected screen is up it is ready for the next job.

In summary:

1. Job ends with success and schedules reboot.
2. Job waits for reboot to complete, fails if timeout elapses.
3. Job finishes tests.

> * Created a build-flow name " Start test"
>
> * After that created 3 freestyle projects.
>
> * Project1 : Before Reboot
>
> * Command: echo "Before Reboot"
>
> * Project 2 : Reboot
>
> * Command : shutdown /r /t 0
>
> * Project 3 : After reboot
>
> * Command : echo "after reboot"
>
> * Then run the above 3 projects from build-flow
>
> * The "Start test" project fails after project 2

f-4875-92b0-30f0c8dbbe04%40googlegroups.com > <https://groups.google.com/d/msgid/jenkinsci-users/d2f0f06b-5d
cf-4875-92b0-30f0c8dbbe04%40googlegroups.com?> utm_medium=email&utm_source=footer> .

> For more options, visit
> https://groups.google.com/d/optout.
>
>
>
>
> --
> You received this message because you are subscribed to
> the Google Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving
> emails from it, send an email to
> jenkinsci-use...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/d5548546-f61

d-4476-981c-245669283299%40googlegroups.com > <https://groups.google.com/d/msgid/jenkinsci-users/d5548546-f6
1d-4476-981c-245669283299%40googlegroups.com?> utm_medium=email&utm_source=footer> .

> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the
> Google Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to jenkinsci-use...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtFPK1P
P6Rsx1BhEc8t1ZKgN3McXSvQ5zF9DoWDD33Hkgg%40mail.gmail.com > <https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtFPK1
PP6Rsx1BhEc8t1ZKgN3McXSvQ5zF9DoWDD33Hkgg%40mail.gmail.com?> utm_medium=email&utm_source=footer> .

> For more options, visit https://groups.google.com/d/optout.
>
>


--

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.