Cannot run agent in MSYS2

[Joel Bodenmann]

I've been testing GoCD today. One of my main requirements is that the Windows agent(s) can perform builds within MSYS2 (unix environment).

To get started I've opened an MSYS2 / MinGW64 terminal and pasted the "Test Drive GoCD" line in there. Everything worked out of the box. The script properly setup both the server and the agent and everything was running within MSYS2. I was able to perform builds as per my requirements.

The next step was to only get the agent running as the server itself is running on a separate FreeBSD machine. However, when attempting to run the agent within MSYS2 (MinGW64 shell) I'm presented with this message:

$ ./bin/go-agent console

Unable to locate any of the following binaries:

  /home/joel/go-agent-20.10.0/bin/../wrapper/wrapper-mingw64_nt-10.0-19042-x86-64

  /home/joel/go-agent-20.10.0/bin/../wrapper/wrapper-mingw64_nt-10.0-19042-x86-32

  /home/joel/go-agent-20.10.0/bin/../wrapper/wrapper

To me it seems that the java server wrapper is not finding the binaries for my particular environment. However, the "Test Drive GoCD" script/binaries ran without any issues.

How can I solve this problem? Ultimately I just need to run the go-agent within MSYS2.

[Jason Smyth]

Hello,

I'm not familiar with MSYS2 so maybe I am missing an obvious point but why is it necessary that the Agent process itself run within MSYS2?

Can the Agent process not just launch a child MSYS2 process if the Job requires it?

Regards,

Jason

[Joel Bodenmann]

Jason,

Thank you for your reply.

I am completely new to GoCD. If I can configure the agent to run all tasks within the MSYS2 / MinGW64 shell then things will work and the agent itself does not need to run inside MSYS2.

The reason I attempted to run it inside MSYS2 is because I noticed that then all tasks executed by the agent are executed in the MSYS2 / MinGW64 shell without any further configuration.

If I'm able to configure the agent to run each task in the MSYS / MinGW64 shell I'm more than happy. How can I do that?

Best regards,

~ Joel

[Jason Smyth]

Hi Joel,

I can't say for sure that it is possible because I am not familiar with MSYS2.

GoCD's Task is effectively a wrapper around a command that gets passed to (I think) cmd.exe on Windows. Assuming there is a way to launch MSYS2 (and pass it an instruction set) from the command line it should be fairly straight-forward to configure GoCD to launch it as a child process and execute whatever commands need to be executed in that environment.

Sorry I can't be more specific. I don't know the specific technology you depend on but the high-level concept I have in mind whenever I need to get GoCD to do something new is "I can make it do anything I can make the computer do from the command line."

Hope this helps,

Jason

[Sushma Gangaiah]

Hi All,

Very oftenly pipeline cannot be triggered and material update in master will get hunger for long.

How can I resolve this?

Regards,

Sushma

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/027ae335-8701-4d90-afe9-09558a961c57n%40googlegroups.com.

[Joel Bodenmann]

Hello Jason,

Unfortunately I made no progress on this. Furthermore, this is the only thing I have to solve/get working before I can start transitioning to GoCD.

The issue is exactly that the custom task command gets passed to `cmd.exe` on Windows.

Before continuing going down the rabbit hole: Is there a way within GoCD to specify the terminal/shell to use for executing tasks? Something such as `SHELL_BIN`?

I am able to issue commands on cmd.exe which get executed in the MSYS2 bash. However, I have to do this for every command (eventually GoCD task). You mentioned the possibility to launch the MSYS2 shell from cmd.exe (easy) as a child process. Once the shell is up, how would I instruct GoCD to use that child process shell?

Best regards,

~ Joel

[Jason Smyth]

Hi Joel,

There is no way that I know of to specify a particular interpreter for GoCD Tasks but it isn't something I have ever looked into so it might be possible.

You are correct that each Task needs to explicitly invoke the child shell of choice. We use PowerShell rather than MSYS2 bash, but I suspect many of the concepts are the same.

The way we avoid having to do that for every command is with PowerShell scripts that we store in source control. Instead of having each command explicitly called from its own GoCD Task, a single GoCD Task launches a PowerShell script that does any number of things. This is nice because it eliminates a lot of otherwise redundant configuration in GoCD (how many times do I really want to type "powershell.exe -NonInteractive" into GoCD?). It also creates a reasonable boundary between our build/deploy processes (the scripts) and the orchestration tool (GoCD). If at some point we should decide that we want to look at other CD solutions, we already have all of the code necessary to execute our processes in a portable format.

If scripting isn't a viable option, I suggest comparing the filesystems between the working "Test Drive GoCD" environment and the broken agent-only environment. Maybe something will jump out as an obvious solution to the apparently missing wrapper problem.

Hope this helps,

Jason

[Aravind SV]

Hello,

I completely agree with Jason about creating that boundary.

About the difference between the server and the agent, I’m not sure, but did you try downloading the zip “installer” option?

Regards,
Aravind

[Joel Bodenmann]

After all considerations, I think the best solution (for me) is to go with the original idea to run the agent directly within the MSYS2 environment. This would solve a lot of problems and not require me to deal with all the extra fuzz.

To clarify:

• The "Test Drive GoCD" bash script for Windows (gocd-20.10.0-12356-1212) can be launched inside of MSYS2 and everything works
• Downloading the agent (go-agent-20.11.0) as the generic ZIP archive leads to the error reported original when attempting to run it in MSYS:

$ ./bin/go-agent console

Unable to locate any of the following binaries:

  /home/joel/go-agent-20.10.0/bin/../wrapper/wrapper-mingw64_nt-10.0-19042-x86-64

  /home/joel/go-agent-20.10.0/bin/../wrapper/wrapper-mingw64_nt-10.0-19042-x86-32

  /home/joel/go-agent-20.10.0/bin/../wrapper/wrapper

To further investigate I also tried to run the GoCD server (generic ZIP: go-server-20.11.0) inside MSYS. The same issue as with the agent is arising: The missing wrapper binaries.

Comparing the filesystems between the plain gocd-agent and the test-drive-suite I see a lot of differences. I'm no Java developer but the wrapper libraries are not present in the test-drive-suite as library files like they are with the plain agent or server. I'm no java developer but I assume that they have been embedded into the test-drive-suite binaries.

I've checked the `try.sh` script that's used by the test-drive-suite to see whether there are some setups happening that allow it to run inside of MSYS2 which are missing when attempting to run just the agent. Unfortunately, I couldn't find anything suspicious. The test-drive-suite has a pre-compiled `run-gocd.exe` binary.

Could anybody tell me where to go from here?

Best regards,

~ Joel

[Sriram Narayanan]

On Sat, Nov 28, 2020 at 6:54 PM Aravind SV <aven...@thoughtworks.com> wrote:

Hello,

I completely agree with Jason about creating that boundary.

Indeed. It is better to move various tasks into a version-controlled shell script/ batch file and to then reference that as a material.

This helps us try the script on a test or our own local machine, keeps us independent of the CI tool, and enables us to orchestrate actions via pipelines with lesser effort (Use a shell script as a material in a different pipeline, rather than move 10 tasks over, for e.g)

-- Ram 

[Ketan Padegaonkar]

I believe the windows installers are stripped off the mysys wrapper libraries to reduce space. You can probably grab the missing files from the test drive binary and put them in your gocd installation directory.

- Ketan

--

You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/cd07a000-b1eb-49ee-84f0-5f1656137b3en%40googlegroups.com.

[Joel Bodenmann]

Ketan,

Unfortunately I have so far been unable to locate the wrapper binaries inside the test drive binary. The test drive ZIP archive appears to contain pre-compiled executables. The Java Service Wrapper binaries are nowhere to be found in there. 

I had a look at the corresponding repository (https://github.com/gocd-contrib/gocd-trial-launcher) but no Java Service Wrapper binaries are to be found in there either.

Is it possible that the test-drive setup uses different means to start GoCD and therefore doesn't require/use the Java Service Wrapper at all? It appears to be a Go (golang) application.

~ Joel

[Fenn]

MSYS2 is a fork of Cygwin. Many things that are true in Cygwin are
also true for MSYS2, but not all. The Goal of Cygwin was to create a
Unix-like / Linux-like environment which runs on Windows - as such is
tries to do everything in Linux-like way with very few exceptions.
MSYS2 has the goal of creating a Unix-like / Linux-like software
development environment for developing native windows binary software
without having to forgo familiar Linux tools. In my personal
experience this boils down to a few concrete differences. MSYS2
provides packages using pacman ported from Arch Linux. Cygwin
provides packages with a Windows Executable called setup.exe (or more
recently setup-x86.exe AND/OR setup-x86_64.exe). The MSYS2 package
manager is meant to be run from the MSYS2 command line and the Cygwin
package manager is meant to be run from Windows, and often requires a
reboot - or in the very least completely shutting down all Cygwin
executable (including daemons like sshd). Python on MSYS2 works well
and is both Windows and MSYS2 aware (and has many patches from the
MSYS2 team to make sure that it works and keep working). Python under
Cygwin is Cygwin awair but not Windows aware - the ctypes packas works
only with linux-like libraries and not well with Windows DLLs (and the
Python Community like it that way and will not accept any Cygwin
specific packages into the main CPython project). The biggest
difference is that MSYS2 linux-like binaries are built for use in the
MSYS2 environment (using and relying on the MSYS2 DLL which offers all
the Linux-like behavior) and also as stand-alone WIndows executables
(32 and 64 bit binaries) which use only standard issues Microsoft
Windows DLLs. The KDE team as adopted MSYS2 and continues to develop
and extend it and uses it to develop the Windows ports of KDE tools,
executables and libraries. I recently installed the KDevelop tool for
Windows and it works quite well as a stand-alone Windows executable.

Some MSYS2 / Cygwin things to be aware of: Java is available
primarily as a "Windows Executable" so, for example, it is not aware
of MSYS2 / Cygwin mounts, paths, or line-endings. Java on Windows
detects that it is running on Windows and assumes all path names are
separated by "\" and not "/" and start with "C:\" and not with "/bin"
or "/usr/bin" (etc.) or "/cygdrive/c/" . It assumes that all text
files have lines delimited with '\r\n' and no file have lines
delimited with '\n'. In mind these are foolish assumption to be made
by a language or standard library, but this is the reality of mixing
MSYS2 / Cygwin with java.

If go-cd agent on Windows is capable of running a binary executable on
Windows (from PowerShell or any other way e.g. The
java.lang.Runtime.getRuntime() method) then it can run the MSYS2
bash.exe. There are three versions of the MSYS2 bash executable, the
one that uses the MSYS2 DLL, and two stand alone windows executables
(one a 32bit binary and one a 64bit binary). The full path for the
executable should be specified with "DOS-slash" and not with
"Unix-slash" such as "C:\MSYS2\bin\bash.exe" , but any arguments given
to it may use "Unix-slash" and probably should. Java and/or windows
will not honor the Linux she-bang so you cannot simply run a script by
name - bash, however will honor it.

Command line arguments with quotes, and nested quotes is another
hurtle to consider. The easiest solution is to use no quotes as all,
and to embed any arguments in the script instead.
Windows. If you really want the details: MSYS2 and Java on Windows
use the Windows CreateProcessA or CreateProcessW DLL method (the one
uses "ascii" which is really Windows-1252 or CP-437 the other uses
"Wide Characters" which is called "Unicode" and is actually something
close to UTF-16LE). The second argument is called lpCommandLine and is
all command-line-arguments combined into one string. The rules of
parsing these arguments into more-than-one argument (which is done by
each executable and not by the shell) and may be done ad-hoc or may
use the windows GetCommandLineA or GetCommandLineW and
CommandLineToArgvW DLL methods but may not.
You can google them for details.

Running the bash executable with a single command line argument of the
full path name of your script should result in the bash script
running. For Example:

C:\msys2\bin\bash.exe /home/jjones/bin/myscript.bash

Once you have managed to run a bash script your bash script can use
"cygpath" to convert back and forth from DOS/Widows filenames and
Unix/Linux stile command lines.
(It can also be used from Java, Jython, JRuby, Groovy etc with some effort.)
The bash script should have all the "Linux-like" behaviors you
associate with MSYS2 / Cygwin.

You could also try using the
https://github.com/gocd-contrib/script-executor-task plugin and see if
it helps at all in your situation. I haven't tried it on
Windows+MSYS2 but we use if for Linux tasks.

For additional help you should find and use the user support forums
for MSYS2 and/or Cygwin.

One last thing:

Linux executables which start scripts, such as cron or login-deamons
set up some basic envronment variables and other environmental
features (current working directory and umask). These are then
inhereted by any subprocesses which are started so they are easy to
assume to be "just there" but may not exist when you are starting
processes in an unusualy way - like AutoSys. In my experience even
cron on different flavors of Linux / Unix may or may not set these
things. I have created the following "template" which I cut-and-paste
to the top of scripts to be run from process automation tools - i call
it my "CRON SAFE START". Here it is:


#!/bin/bash

### https://linux.die.net/man/1/login
### https://linux.die.net/man/5/crontab
### https://unix.stackexchange.com/questions/76354/who-sets-user-and-username-environment-variables

PATH_ORIG="$PATH" ;
export PATH_ORIG ; echo "PATH_ORIG=$PATH_ORIG"
PATH=/usr/local/bin:/bin:/usr/bin ;
export PATH ; echo "PATH=$PATH"
SCRIPT_USERNAME="$(whoami)" ;
export SCRIPT_USERNAME ; echo
"SCRIPT_USERNAME=$SCRIPT_USERNAME"
SCRIPT_UMASK=0022 ;
export SCRIPT_UMASK ; echo "SCRIPT_UMASK=$SCRIPT_UMASK"
LOGNAME="$SCRIPT_USERNAME" ;
export LOGNAME ; echo "LOGNAME=$LOGNAME"
USER="$SCRIPT_USERNAME" ;
export USER ; echo "USER=$USER"
USERNAME="$SCRIPT_USERNAME" ;
export USERNAME ; echo "USERNAME=$USERNAME"
HOME="$( /bin/bash --norc --noprofile -c 'echo ~'"$USER" )" ;
export HOME ; echo "HOME=$HOME"
SHELL=/bin/bash ;
export SHELL ; echo "SHELL=$SHELL"
TERM=dumb ;
export TERM ; echo "TERM=$TERM"
#SCRIPT_LANG="C" ;
export SCRIPT_LANG ; echo "SCRIPT_LANG=$SCRIPT_LANG"
SCRIPT_LANG="en_US.UTF-8" ;
export SCRIPT_LANG ; echo "SCRIPT_LANG=$SCRIPT_LANG"
LANG="$SCRIPT_LANG" ;
export LANG ; echo "LANG=$LANG"
LANGUAGE="$SCRIPT_LANG" ;
export LANGUAGE ; echo "LANGUAGE=$LANGUAGE"
LC_ALL="$SCRIPT_LANG" ;
export LC_ALL ; echo "LC_ALL=$LC_ALL"
#TZ="UTC0" ;
export TZ ; echo "TZ=$TZ"
TZ="America/New_York" ;
export TZ ; echo "TZ=$TZ"

cd "$HOME"
umask "$SCRIPT_UMASK"




Brian Fennell