Using nuget with Go as a custom command

[Ankit]

Hi,

I have just configured my first Go setup, so I might be missing something entirely obvious, but after reading plenty of documentation and blogs I'm struggling to get this simple task to work.

I am trying to build a sole .NET project that is part of a much bigger solution, this is my current configuration for the job itself

<job name="build"> 
<tasks> 
<exec command="D:\GoBinaries\nuget.exe"> 
<arg>restore packages.config</arg> 
<arg>-PackagesDirectory ..\packages</arg> 
<runif status="passed" /> 
</exec> 
<exec command="MsBuild"> 
<arg>DataContracts.csproj</arg> 
<arg>/t:Clean,Build</arg> 
<arg>/p:Configuration=Debug</arg> 
<runif status="passed" /> 
</exec> 
</tasks> 
</job>

The first task is meant to restore any missing nuget packages; the second builds the project.

Here's the console output, indicating that the "restore packages.config" command is not recognised.

[go] Start to execute task: <exec command="D:\GoBinaries\nuget.exe" > 
<arg>restore packages.config</arg> 
<arg>-PackagesDirectory ..\packages</arg> 
</exec>. 
[...] 
[go] setting environment variable 'GO_TO_REVISION' to value '7259' 
[go] setting environment variable 'GO_FROM_REVISION' to value '7259' 
Unknown command: 'restore packages.config'

If I open the Go Agent directory where the files are getting pulled from version control, I have no issues at all running the commands manually from a DOS prompt.

Any ideas on how could I make Go interact with the nuget executable?

Thanks very much,

Jaume

ps: I have seen this article http://support.thoughtworks.com/entries/22507888-How-to-Use-NuGet-With-Go 
but if I understand correctly, Mark is suggesting to put the nuget.exe binary under version control. 
Which is something that I'd rather not do, as that would bind the binary to every project and it would make the upgrade of the nuget executable fairly involved.

[Jaume Sancho]

I have managed to get the command to execute following the "Windows Tips" on this article

http://support.thoughtworks.com/entries/22873043-Go-s-custom-command

Resulting with the following config 

<exec command="cmd">

<arg>/c</arg>

<arg>D:\GoBinaries\nuget.exe restore packages.config -PackagesDirectory ..\packages</arg>

<runif status="passed" />

</exec>

It feels a bit hacky, but for the time being, is good enough for me. 

Any ideas on what could be the issue on running nuget with the standard syntax?

[Rustin Daniels]

Hi Ankit,

I'm wondering why you're not wrapping your command in nant macrodefiitions and have a component.build to execute the target build. It will make things much easier than editing commands for every pipeline as you expand your build and deployment platform.

Versioning your  nuget.exe by placing it the tools folder of your repository would be the right thing todo, then adding tools folder as an upstream dependency to your your build pipeline, this essentially sandboxes your component build from the rest of your pipelines and would pull the files your agent directory in this case your nuget.exe

Running dos command would work fine since all windows machines have %SystemRoot%\system32; in the Path Environment variable, your command="MsBuild" wont work since ${framework::get-runtime-framework()} => msbuild.exe is not in your path environment variable and even then you will need an absolute path to the target solution file and not the project file as your argument.

My personal preference would be to use Nant, here's an example configuration for a common taxonomy of an integration build pipeline. I hope it helps you get started with this great tool.

I still use lib folder structure and not a fan of nuget on the build server, this might seem bit backward but I prefer it that way since I have many diamond dependency references.

 <pipeline name="Framework.Common.Configuration" labeltemplate="1.0.${COUNT}.${target}" isLocked="false">

      <materials>

        <svn url="https://10.194.74.34/svn/Enterprise.Framework/trunk/Framework.Common.Configuration/" username="GoAgent" encryptedPassword="sIK49HOc1rdBuGJ69kfLRw==" dest="target" materialName="target" />

        <svn url="https://10.194.74.34/svn/Enterprise.Framework/trunk/lib/" username="GoAgent" encryptedPassword="sIK49HOc1rdBuGJ69kfLRw==" dest="lib" materialName="library">

          <filter>

            <ignore pattern="**/*.*" />

          </filter>

        </svn>

        <svn url="https://10.194.74.34/svn/Enterprise.Framework/trunk/build/" username="GoAgent" encryptedPassword="sIK49HOc1rdBuGJ69kfLRw==" dest="build" materialName="build">

          <filter>

            <ignore pattern="**/*.*" />

          </filter>

        </svn>

        <svn url="https://10.194.74.34/svn/Enterprise.Framework/trunk/tools/" username="GoAgent" encryptedPassword="sIK49HOc1rdBuGJ69kfLRw==" dest="tools" materialName="tools">

          <filter>

            <ignore pattern="**/*.*" />

          </filter>

        </svn>

      </materials>

      <stage name="defaultStage">

        <jobs>

          <job name="defaultJob">

            <tasks>

              <nant nantpath="tools\Nant" buildfile="target\Component.build" target="CompileAndTest" />

            </tasks>

            <resources>

              <resource>Auto</resource>

              <resource>Integration</resource>

            </resources>

            <artifacts>

              <artifact src="target\BuildArtifacts\Deploy\Framework.Common.Configuration.dll" dest="lib" />

            </artifacts>

          </job>

        </jobs>

      </stage>

    </pipeline>

  </pipelines>

Regards

Rustin

[Jaume Sancho]

Hi Rustin, 

I'm the original poster here, Ankit sent the message on my behalf since I sent the request to the wrong place.

Yes, I think you are right with the idea of putting all the tools in their own folder under source control.

As that will allow us to reference all the executables from a single path reference, as well as have them automatically update on all agents whenever we upgrade one tool.

For the time being, I'm trying not to use nAnt if I can help it. Partly because it does increase the complexity of the build for any newcomers, and partly because the tasks are simple enough to be defined with "raw" commands, e.g. msbuild, mstest, msdeploy, etc.

But the fact that nAnt is natively supported by Go might make it appealing in the future if the tasks grow bigger, as I'm guessing that Go will handle nicely the propagation of variables and errors.

Thanks for the tips and your sample script, it is rather handy as a reference!

Regards,

Jaume

[Marius Ciotlos]

Hey, 

Looking at the orignal post one thing to notice is this:

<arg>restore packages.config</arg> 

In go to have those sent to the <exec> command as parameters you would need to add each parameter in one new line, otherwise it would not work correctly. Go does some escaping for parameters. 

Cheers, 

Marius

[Jaume Sancho]

Hi Marius, 

You are absolutely right.

When I originally said "I might be missing something entirely obvious" that is exactly what was happening.

Go takes each argument with a new line. So all I *really* needed to do, was break it down like this:

<tasks> 
<exec command=#{nuget}"> 
<arg>restore</arg>

<arg>packages.config</arg> 

<arg>-PackagesDirectory ..\packages</arg> 
<runif status="passed" /> 
</exec> 

So thanks very much indeed, I'm now starting to get the hang of this beast.

Regards,

Jaume

--
You received this message because you are subscribed to a topic in the Google Groups "go-cd" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-cd/5-1pP8cEsQk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to go-cd+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Carl Reid]

Hi Jaume

I cannot help you with nuget at the moment as we are currently not using it in the way you describe, we simply install nuget packages within the Visual Studio solution and store all the packages in source control so they are available to our projects at compile time without requiring any references to nuget at all. We will probably look to move away from this in the future however for now it works well enough,

However from your description if what you are trying to achieve in your environment is most probably very similar to ours and we have been on this journey for 6 months or so. Therefore if you have questions on using GO with .NET tools such as MSBuild, MSDeploy etc I may be able to help. Feel free to drop me a line or continue to use this forum which I look at quite often.

Thanks

carl

[Jaume Sancho]

Hi Carl, 

This is a rather old question now. 
The actual issue was more related to me getting to grips with breaking down the arguments one-per-line than anything to do with nuget itself.
The previous reply to this thread shows how I finally managed to get it done.

I have come a long way automating things with Go, and we are now gotten to the point of pushing changes to production even.

The one remaining task that still involves some manually processing is updating SQL Server. 

We have a pipeline for the DB projects themselves, and nightly deployments to a DEV server.

However, I still haven't had the chance to tackle the issue of doing incremental SQL upgrades. i.e. udpating the schema without losing any existing data.

Is that something that you have done at your end? Care to share any tips or gotchas that you might have found?

thanks very much for chipping in, I really appreciate it.

Jaume

[Sriram Narayanan]

On Fri, Sep 5, 2014 at 4:05 PM, Jaume Sancho <j...@ume.me.uk> wrote

<snip/>

The one remaining task that still involves some manually processing is updating SQL Server. 

We have a pipeline for the DB projects themselves, and nightly deployments to a DEV server.

However, I still haven't had the chance to tackle the issue of doing incremental SQL upgrades. i.e. udpating the schema without losing any existing data.

Is that something that you have done at your end? Care to share any tips or gotchas that you might have found?

I have used RedGate's SQLCompare for this. Amazing tool.

-- Ram

[Jaume Sancho]

Hi Sriram, 

We are currently using the SQL Schema compare available as part of the SQL Server Data Tools for Visual Studio 2013.

However, I was referring to integrating this upgrade as part Go's pipelines.

Ideally I would also like to perform this step with free tooling, even if it requires some additional setup.

So far I have considered using either VSDBCMD or http://dbdeploy.com/ , but I haven't had the time to go into much detail.

If anyone has any ideas or past experiences to share on that front, that would be much appreciated.

Thanks,

Jaume

[Sriram Narayanan]

I just read : http://msdn.microsoft.com/en-us/library/dd193283%28v=vs.100%29.aspx

Seems to me you should be able to do what you want to do from the Command Line, and trigger it all via Go.

-- RAm