TFS Checkout Fails After Updating from 15.2 to 18.10

26 views
Skip to first unread message

Jason Smyth

unread,
Nov 2, 2018, 4:27:31 PM11/2/18
to go-cd
While testing an update from version 15.2 to 18.10 we have run into an issue that may be bug in either GoCD or the TFS JDK that it uses.

For reasons that I won't go into, we have several pipelines that include multiple TFS materials, some of which are single files from our TFS repository. For example:

  • AssemblyInfo
    • Project Path: $/TeamProjectName/AssemblyInfo.cs
    • Destination Directory: tfs/AssemblyInfo.cs
  • Packages:
    • Project Path: $/TeamProjectName/packages
    • Destination Directory: tfs/packages
We have to include the file name in the AssemblyInfo material's Destination Directory (DD) because otherwise the Packages material's DD appears to be nested inside the AssemblyInfo DD and GoCD throws an error when trying to save the configuration.

On version 15.2 everything works as intended but after updating the GoCD agent fails to check out the AssemblyInfo material with an error like the following:

Failed while checking out into Working Folder: pipelines\MyPipeline\tfs\AssemblyInfo.cs, Project Path: $/TeamProjectName/AssemblyInfo.cs, Workspace: bb18a58c6abfa36db84643de09258c86feaa48bc711174334eaea61594b9623f, Username: BuildUser, Domain: mydomain, Root Cause: java.io.FileNotFoundException: E:\GoCD\Go Agent\pipelines\MyPipeline\tfs\AssemblyInfo.cs\$tf\5\377de3f5-a6f8-41ed-970a-9b0bbf2c1a50.gz (The system cannot find the path specified)
 
This error is consistent across all tested pipelines that follow this format.

Based on the path of the file that is triggering the FileNotFoundException, it seems as if the agent is attempting to create the $tf directory required by a local TFS workspace inside the file that it is trying to check out. Since you can't create a directory inside a file, this fails and eventually results in the FileNotFoundException being thrown when the checkout process tries to access a file from the $tf directory.

It would seem that an easy fix would be to tell the agent to use a server workspace instead of a local one but I haven't been able to find a way to configure that option.

Does anyone have any experience with this issue or any suggestions on how we can overcome it?

In case it's relevant we are running GoCD server on a 64-bit Linux system and the agent is Windows Server 2012R2 64-bit running the agent bundled with 32-bit Java.

Thanks in advance for any advice you can offer,
Jason

Aravind SV

unread,
Nov 6, 2018, 4:53:53 AM11/6/18
to go...@googlegroups.com
Hello!

Don't know much about this, but what was happening in the old version, with this setup? What was the directory structure of the checkout like?

I must say I'm a little surprised that providing a file for "Destination Directory" was ever working.

Cheers,
Aravind

On Fri, Nov 02, 2018 at 13:27:31 -0700, Jason Smyth wrote:
> While testing an update from version 15.2 to 18.10 we have run into an
> issue that may be bug in either GoCD or the TFS JDK that it uses.
>
> For reasons that I won't go into, we have several pipelines that include
> multiple TFS materials, some of which are single files from our TFS
> repository. For example:
>
>
> - AssemblyInfo
> - Project Path: $/TeamProjectName/AssemblyInfo.cs
> - Destination Directory: tfs/AssemblyInfo.cs
> - Packages:
> - Project Path: $/TeamProjectName/packages
> - Destination Directory: tfs/packages
> --
> You received this message because you are subscribed to the Google Groups "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Jason Smyth

unread,
Nov 6, 2018, 9:41:50 AM11/6/18
to go-cd
Hello Aravind,

Thank you for taking the time to reply.

Under version 15.2 this is working as intended: the AssemblyInfo.cs file is checked out to "./tfs/AssemblyInfo.cs" and the packages directory and all of its contents are checked out to "./tfs/packages", where "." is the pipeline working directory on the agent.

We use this setup because it allows us to maintain the relative paths between source locations without having to check out the entire Team Project.

Any insight on this is greatly appreciated. As of now it is a blocker for us updating our GoCD server from its current 3-year old version.

Thanks,
Jason

Ketan Padegaonkar

unread,
Nov 6, 2018, 9:56:55 AM11/6/18
to go...@googlegroups.com
Could you perhaps paste a snippet of the <materials/> XML element in your config for us to check what's up?

- Ketan



Jason Smyth

unread,
Nov 6, 2018, 10:29:50 AM11/6/18
to go...@googlegroups.com

Hi Ketan,

 

I created a simple pipeline that runs as a test. Here is the full XML for that pipeline:

 

    <pipeline name="Test-FileCheckout">

      <materials>

        <tfs url="http://tfs.mydomain.com:8080/tfs/DefaultCollection" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/AssemblyInfo.cs" dest="tfs/AssemblyInfo.cs" materialName="AssemblyInfo" />

      </materials>

      <stage name="defaultStage">

        <jobs>

          <job name="defaultJob" />

        </jobs>

      </stage>

    </pipeline>

 

This pipeline runs successfully under version 15.2 but file checkout fails under version 18.10.

 

Please let me know if there is any additional information I can provide.

 

Regards,

Jason

--
You received this message because you are subscribed to a topic in the Google Groups "go-cd" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-cd/00PZ6loeunk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to go-cd+un...@googlegroups.com.

Jason Smyth

unread,
Nov 6, 2018, 3:58:37 PM11/6/18
to go...@googlegroups.com

Here is a slightly modified version of the materials element for one of our live pipelines that demonstrates why, for us, including a file name in the dest attribute is sometimes necessary to allow proper relative file paths:

 

      <materials>

        <tfs url="http://tfs.mydomain.com:8080/tfs" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/Products/MyGreatProduct/Trunk" dest="tfs/Products/MyGreatProduct/Trunk" materialName="tfs" />

        <tfs url="http://tfs.mydomain.com:8080/tfs" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/AssemblyInfo.cs" dest="tfs/AssemblyInfo.cs" materialName="AssemblyInfo" />

        <tfs url="http://tfs.mydomain.com:8080/tfs" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/CommonServices" dest="tfs/CommonServices" materialName="CommonServices" />

        <tfs url="http://tfs.mydomain.com:8080/tfs" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/ThirdParty" dest="tfs/ThirdParty" materialName="ThirdParty" />

        <tfs url="http://tfs.mydomain.com:8080/tfs" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/packages" dest="tfs/packages" materialName="packages" />

        <tfs url="http://tfs.mydomain.com:8080/tfs" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/Products/MyGreatProduct/Documents/Spec" dest="tfs/Products/MyGreatProduct/Documents/Spec" materialName="Spec" />

        <tfs url="http://tfs.mydomain.com:8080/tfs" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/Internal/BuildTargets" dest="tfs/Internal/BuildTargets" materialName="BuildTargets" />

        <tfs url="http://tfs.mydomain.com:8080/tfs" username="myServiceAccount" domain="mydomain" encryptedPassword="REDACTED" projectPath="$/MyTeamProject/Internal/BuildScripts" dest="BuildScripts" materialName="BuildScripts" />

      </materials>

 

Again, any insight on this is appreciated.

 

Thanks,

Jason

Jason Smyth

unread,
Nov 20, 2018, 5:29:01 PM11/20/18
to go-cd
I created a bug on GitHub because it turns out that it actually isn't relevant whether or not Destination Directory contains the file name to be used or not; TFS checkout just straight up fail if the Project Path is a single file rather than a directory.

I still feel that this can be solved by telling the underlying TFS agent to create a server workspace instead of a local one but I have no idea how to do that.

If anyone has any ideas, I'd love to hear them.

Regards,
Jason

To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscribe@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "go-cd" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-cd/00PZ6loeunk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to go-cd+unsubscribe@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "go-cd" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-cd/00PZ6loeunk/unsubscribe.

To unsubscribe from this group and all its topics, send an email to go-cd+unsubscribe@googlegroups.com.

Reply all
Reply to author
Forward
0 new messages