Pipeline powershell

[jer...@bodycad.com]

Hi,

I just realized that the powershell is not part of pipeline script. I'm no powershell guru. So I wonder how one can use it properly to launch something like msbuild (I known I can launch it with the bat but this often hang like many jenkins issues have already show the problems). So I hope to have something finally working with it, so we can put the Jenkins Ci into a real life CI usage. So my questions are the following:

1. Is it better to execute a pregen scripts myScriptFile.ps1 ?
2. How can one invoke the following batch into powershell properly:
   bat returnStatus: false, script: "\"${bcad.msbuild_current}\" ${bcad.msbuild_solution_name} ${bcad.msbuild_default_arg} /t:Build"
   
   
   

Do I have to 

Invoke-Expression

or do I have to force escape with 

--%

Thanks

Jerome

[jer...@bodycad.com]

So far I can execute the following into a powershell:

$msbuild = "C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe"

$options = "BodyCad.sln /m /p:Configuration=Release;Platform=x64;SolutionDir=`" + $PSScriptRoot + `"\ /t:Clean;Build"

Invoke-Expression "& '$msbuild' $options"

But I cannot make this happen when running into piepline powershell so far

[jer...@bodycad.com]

def psScript = """
\$msbuild = "${bcad.msbuild_current}"
\$options = "${bcad.msbuild_solution_name} ${bcad.msbuild_default_arg.replace('"', '`"').replace('%CD%', '$PSScriptRoot')} /t:Build"
Invoke-Expression "& '\$msbuild' \$options"
""";
powershell returnStatus: false, script: psScript

is doing the trick, have to replace the " and the %CD% which was a batch path replacement. Now it seem to work. Will make it compile a lot over the weekend to see if this hang as much as batch was.

[Slide]

Why not use the msbuild plugin to launch msbuild? https://wiki.jenkins.io/display/JENKINS/MSBuild+Plugin?focusedCommentId=67568742

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f1427e69-e1fa-47c6-92c8-dd616b7d38c7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jer...@bodycad.com]

unless I'm mistaken or I missed the feature, MsBuild plugin is not pipeline compatible (https://github.com/jenkinsci/pipeline-plugin/blob/master/COMPATIBILITY.md seem to agree on that, not sure if this up to date). If I'm wrong I will totally up to use it. I was using batch because the plugin was not compatible, now I use powershell because the batch often hang when the MsBuild command terminate.

so yeah this is a workaround the other workaround to just make this simple use case work :-(

[Slide]

You're right, I was looking at the tool descriptor having an @Symbol.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/45182aaf-8128-4bcc-8c3b-3a435240d6c4%40googlegroups.com.

[James Nord]

You can you use powershell natively in pipeline now.
https://issues.jenkins-ci.org/browse/JENKINS-34581

[Jérôme Godbout]

Not even 3 triggered build did terminate, it's still hanging itself. So the powershell is no better then batch into this regard.

This is getting ridiculous, a CI system who hang is worthless. I guess it's time to find an alternative.

Jerome Godbout Software Developer 2035 rue du Haut-Bord, Québec, QC, Canada. G1N 4R7 T:  +1 418 527-1388 E: jer...@bodycad.com www.bodycad.com

The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.  Le contenu de ce message et les pièces jointes sont destinés uniquement pour le(s) destinataire(s) et peuvent contenir des informations confidentielles et / ou privilégiées qui peuvent être protégées légalement contre toute divulgation. Si vous n'êtes pas le destinataire de ce message ou son agent, ou si ce message vous a été adressé par erreur, s’il vous plaît avertir immédiatement l'expéditeur par courriel de réponse, puis supprimer ce message et les pièces jointes. Si vous n'êtes pas le destinataire prévu, vous êtes par la présente informé que toute utilisation, diffusion, copie, ou stockage de ce message ou de ses pièces jointes est strictement interdit.

On Sat, Jul 1, 2017 at 6:45 AM, James Nord <james...@gmail.com> wrote:

You can you use powershell natively  in pipeline now.
https://issues.jenkins-ci.org/browse/JENKINS-34581

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/vXnWWwss23A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e52b95c4-4aa0-4799-a87b-fc5b882ab9f9%40googlegroups.com.

[Slide]

There are many many people using Jenkins with no issues, so perhaps you could give some more information to help solve your problem rather than just leaving for something else. 

On Tue, Jul 4, 2017 at 6:05 AM Jérôme Godbout <jer...@bodycad.com> wrote:

Not even 3 triggered build did terminate, it's still hanging itself. So the powershell is no better then batch into this regard.

This is getting ridiculous, a CI system who hang is worthless. I guess it's time to find an alternative.

Jerome Godbout Software Developer 2035 rue du Haut-Bord, Québec, QC, Canada. G1N 4R7 T:  +1 418 527-1388 E: jer...@bodycad.com www.bodycad.com

The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.  Le contenu de ce message et les pièces jointes sont destinés uniquement pour le(s) destinataire(s) et peuvent contenir des informations confidentielles et / ou privilégiées qui peuvent être protégées légalement contre toute divulgation. Si vous n'êtes pas le destinataire de ce message ou son agent, ou si ce message vous a été adressé par erreur, s’il vous plaît avertir immédiatement l'expéditeur par courriel de réponse, puis supprimer ce message et les pièces jointes. Si vous n'êtes pas le destinataire prévu, vous êtes par la présente informé que toute utilisation, diffusion, copie, ou stockage de ce message ou de ses pièces jointes est strictement interdit.

On Sat, Jul 1, 2017 at 6:45 AM, James Nord <james...@gmail.com> wrote:

You can you use powershell natively  in pipeline now.
https://issues.jenkins-ci.org/browse/JENKINS-34581

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/vXnWWwss23A/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e52b95c4-4aa0-4799-a87b-fc5b882ab9f9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAKmu%2Be3CMprjPziDGfjJ-xgxgJt4%2BRtgA83ByYBrpuQCO3qY3A%40mail.gmail.com.

[jer...@bodycad.com]

I'm not alone into this situation and this have been a problems for many times for some:

• https://issues.jenkins-ci.org/browse/JENKINS-28759
  
• https://issues.jenkins-ci.org/browse/JENKINS-33164
  
• https://issues.jenkins-ci.org/browse/JENKINS-42988
  

And I probably miss some other, but some are open since 2015 and still are unresolved. I was told this won't be fix and I should look forward to powershell api instead that should not have this :-(

I try to offer to test it, even gave an article that IMO seem to point to a probable problems with the way the process is spawn and check, but seem like I was lost and it was irrelevant. 

http://www.javaworld.com/article/2071275/core-java/when-runtime-exec---won-t.html

Looking at the source, this seem like the pitfall of Java process reading was not avoid, might be at a lower level into CommandInterpreter but since I'm no Java expert (more a C/C++/Python/Bash dev):

https://github.com/jenkinsci/jenkins/blob/d111e2ac1658c8fa5fb768e7d1233613b4b9992d/core/src/main/java/hudson/tasks/BatchFile.java

We have a broken CI for months now (ever since the master is no more a single Windows machine master/slave, now we have a Linux Master and a Windows slave, this behavior is appearing nearly every day, have to restart the Jenkins service, cannot even cancel it via the web GUI). I now need the multiple slave and my master on Linux and it ain't working with msbuild batch nor powershell.

I don't really have time to do this migration, but at some point my team and I need my CI to be reliable, not something that hang every day where I need an admin to log on the machine and restart Jenkins service since we have a master and a slave hang. Really wish this would be resolved by now, but I'm no longer holding my breath and it's a show stopper for us.

Thanks,

regards,

Jerome

[James Nord]

Given you can reliably reproduce if can you open a new ticket with the minimum steps to reproduce (this likely includes an msbuild project) and include your plugin versions, build logs and a stack trace then it is more likely to get fixed - if you can attach a support bundle to the JIRA that will also go a long way in helping the dignosis

Many of the issues have no steps to reproduce and when we try to reproduce the issue we just can not, some of them we believe are just duplicates of JENKINS-34150 (fixed) but when the reporters neither confirm or deny this the issue can stay in limbo.  

[jer...@bodycad.com]

Hi James, 

I will try to install the plugin and provide more information (I was giving the dump and the system info, but I may generate even more with this it seem).

As for JENKINS-34150 goes it definitely did not fix everything, since many bug were open since then with up to date configuration. So you can assume there still is something wrong there without a doubt.

As for giving a working example, I wonder how I can manage to do this properly. I need a master and a slave to do so, I cannot provide the whole System image (there is confidential data and configuration won't work else where with network drive).

I can provide most of the build script, but I cannot give access to the real .sln and .vxcproj files and sources. If you have an idea how I could do the reproduction master/slave setup and all data for reproduction that could be used. This doesn't show up when master/salve was a single Windows node, this only show up with my Linux master inside a Hyper-V and Windows slave Desktop normally installed. I haven't test any other master/node configuration so far. If you have any wish list or way I could give you back the proper usable information to reproduce this that you could use let me know.

I wonder if you could provide a jenkins.war with more debug trace I could try to reproduce the bug on my side (probably not, but worth a try to ask for it)?

Else I will try the following:

1. Make a dummy project
2. make a strip down jenkinsfile script
3. try to run it until it hang
4. make a dummy msbuild .sln, .vcxproj compilation
5. drop the information with the plugin above
6. Submit all this.

The #2 and especially #4 can be a headache to make something that look like the real deal and can reproduce the problems since it seem to occur when the bat command is really long. But if this can reproduce it, I will post it for both bat and powershell.

Will give some feedback as soon as I can.

Jerome

[jer...@bodycad.com]

I have add the support info inside the following bug

https://issues.jenkins-ci.org/browse/JENKINS-42988

As far as the mini repro project, I cannot reproduce it with a strip down pipeline file that does the same bat command on the same repos checkout. So it look like (not sure yet, only 5 build done so far) a particular context, project settings or something outside the bat command make a difference.