Issues with backups on Go 22.2 and PostgreSQL v12

[Funkycybermonk]

Hello!

New upgrade going from GoCD 20.1 to 22.2 and migrated to PostgreSQL v12 on Windows. Everything is now working properly but the backups were blowing up due to not being able to find pg_dump. I added the bin for PostgreSQL to the system path and the backup exits with a status code 1. I went back and checked the command and it looks like it might be blowing up because the dbname isn't defined with a parameter name. 

The executing bit is showing [pg_dump, --no-password, --host=localhost, --port=5432, --username=<removed>, gocd, --file=C:\Program Files (x86)\Go Server\artifacts\serverBackups\backup_20221101-215455\db.gocd] with environment {PGPASSWORD=<removed>}

The gocd bit in the middle appears to be the dbname but its not qualified as --dbname. Is that a bug or is there something I haven't configured properly? If I take that parameter string and run it in a command prompt with the commas removed it blows up on the parameters not matching up. Adding the --dbname= to that parameter works for me but I'm not sure if that should be required. 

I can roll back to H2 but I was hoping there would be a performance improvement and future-proofing if I went ahead with the migration now. If that doesn't make sense then I'm happy to stay with H2.

Thanks!

[Ketan Padegaonkar]

What version of pg_dump are you using? According to https://www.postgresql.org/docs/current/app-pgdump.html, the command line string looks OK. But we've not tested this on windows.

- Ketan

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/1a7de3d6-8dae-4da0-b675-6d02ab885993n%40googlegroups.com.

[Funkycybermonk]

It looks like the file version is 12.0.12.22220.  I'm using the version that shipped with the PostgreSQL 12 installer. I can downgrade to a lower PostgreSQL version if this is an issue with this release of pg_dump.

The message I get when constructing the command line using as-is from the logs is:

"C:\Users\removed>pg_dump --no-password --host=localhost --port=5432 --username=removed gocd --file="C:\Program Files (x86)\Go Server\artifacts\serverBackups\backup_20221101-202201\db.gocd"
pg_dump: error: too many command-line arguments (first is "--file=C:\Program Files (x86)\Go Server\artifacts\serverBackups\backup_20221101-202201\db.gocd")
Try "pg_dump --help" for more information."

I can make it work by adding the --dbname= in front of the gocd value although the directory has to be created before the path from the logs will work. I'm assuming thats because the process failed in GoCD so the folder was removed as cleanup. Creating the folder results in a successful db.gocd backup file being created.

Thanks!

[Chad Wilson]

Technically speaking, the docs say the dbname without --dbname= should be the last argument after all connection options, which GoCD doesn't respect. Possibly pg_dump is more flexible on other OSes than Windows?

If you move "gocd" to the end of the command line you will probably find it works.

If you can raise an issue with the specific details to make it easy to reproduce (I don't work with Windows much, so the easier the better) at https://github.com/gocd/gocd/issues it can probably be fixed/addressed with `--dbname=` prefix to be unambiguous.

You could try other versions of pg_dump and see if the behaviour is different as well.

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/991c4de4-b112-4fc9-a8ed-4a8e3a2bdd3an%40googlegroups.com.

[Chad Wilson]

With a bit of quick pg_dump testing on Windows (via choco install postgresql12/choco install postgresql14) without actually integrating with GoCD) I can confirm the utility complains about the order of the positional (non-option) dbname arguments in ways that the same utility on MacOS does not. Only checked v12 and v14.

So it seems we need to fix something on the server here. I have a draft fix at https://github.com/gocd/gocd/pull/10982 which should be easy to get into the next 22.3.0 release (we were planning to release shortly anyway), especially if you are willing to help with sanity testing backups on your trial Windows setup you are working on.

-Chad

[chan...@gmail.com]

Sure! I’d be happy to drop it in and see if that works! I can double check on the folder creation that way as well. I haven’t entirely worked out building the package myself so if you have an installer I can run or just the files to replace and instructions on where they need to go I’m happy to give it a run.

 

Thanks!

--
You received this message because you are subscribed to a topic in the Google Groups "go-cd" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-cd/AMtgQQS3Gck/unsubscribe.
To unsubscribe from this group and all its topics, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CAA1RwH_cbCo-gjqtjv8tfheiuUDGTWwyQC5iJR3H2izX3DfOuA%40mail.gmail.com.

[Chad Wilson]

Great, thanks Chantry!

An experimental code-signed server installer version is available at https://download.gocd.org/experimental/binaries/22.3.0-15295/win/go-server-22.3.0-15295-jre-64bit-setup.exe  (check experimental tab here if you want to verify checksums etc)

The Windows installers allow upgrades but prevent downgrades (perhaps for some historical reason I am not aware of), so keep that in mind. There aren't any DB or config.xml format changes in this release in any case so should have strong backward compatibility with your 22.2.0 installation, and agents will auto-upgrade/downgrade where necessary to match the server version so there should be no need to also install an experimental agent version.

You can preview the release notes at https://gocd.github.io/pr-review.gocd.org/pr-1416/releases/ if you have any concerns about the other included changes since the 22.2.0 you are testing with.

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/04a101d8eee2%24108c7650%2431a562f0%24%40gmail.com.

[Funkycybermonk]

Backup works great! 

Do you know an eta on the release version or is this the version that will be rolled to release? I'm just weighing time vs using H2 until this is released for my upgrade window. 

Thanks!!

[Chad Wilson]

Great - thanks for getting back to me.

Likely in the next couple of days, with basically the same code as that experimental release (except tagged as stable, thus without the -xxxxx suffix).

-Chad

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/03ad20e4-7d26-4734-b915-c8d6307770cfn%40googlegroups.com.