How to clear cached material source repo on gocd server?

[Bojan S.]

Hi all.

Source repo for 3rd party lib I am using has forced pushed some changed to master, which our gocd accepted, but they removed those commits back to pre force pushed state.

Now, my pipeline is stuck at "new" forced pushed state, and it cannot collect latest master, which is basically old one.

How can I clear git cache on gocd server to use current master state always without getting stuck on git repo illegal state?

Kind regards

B

[Ashwanth Kumar]

Make a new commit to that repo. It should fix the problem. GoCD stores the state in the DB. 

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/629dc34a-2f02-4479-8cd9-b257b7ad247fn%40googlegroups.com.

[Bojan S.]

As I explained, it is 3rd party repo. I cannot make commit to source I do not own.

Please read the original note. Thank you.

[Jason Smyth]

Hi Bojan,

Could you elaborate on what the impact is to your process?

It seems to me that if the source repo is not changing (no updates have been committed), then there should be nothing to build and it wouldn't be a problem that GoCD is "stuck" on the new version of master.

I must be missing something on why this is an issue you need to solve.

Regards,

Jason

[Bojan S.]

Hi, Jason,

problem is that someone forced pushed on master on 3rd party source repo.

GoCD collected those commits, made build with our source, all was fine.

Few days later, someone overwritten master. GoCD is stuck looking for commit XYZ that was forced pushed, and does not collect latest master.

Material for that 3rd party looks like below:

{

"branch": "master",

"destination": "z3",

"name": "z3",

"type": "git",

"url": "https://github.com/somecompany/library"

}

I "fixed" buy changing url to e.g. "https://github.com/somecompany/library.git", and pushed, which works fine, but because of our other libs, this issues long build for every component, plus makes unnecessary commits to our code base.

Please let me know if you need more details.

Kind regards,

Bojan

[Jason Smyth]

Hi Bojan,

I think I understand now. The issue is that there is a Pipeline (or Pipelines) in GoCD that builds multiple source materials, including this 3rd party library. As a result, because GoCD sees the 3rd party library source as "broken", the Pipeline doesn't trigger or doesn't build properly.

Since GoCD stores state in its database, I think there are a few ways you can deal with this situation:

1. Restore GoCD from a backup taken before the now-missing commit was introduced to the 3rd party library source. The obvious downside here is that you would lose anything that happened in GoCD since the backup was taken. Depending on the specific processes you have implemented this may or may not be a significant concern.

2. Manually trigger each affected Pipeline and specify the older version of the library's material. Keep doing this until a new commit is pushed into the library, which should resolve the current issue.

3. Change the material URL so that GoCD believes it is a new material. (Sounds like you have done this already but are unhappy with the result.)

4. Try to "hack" the GoCD database so that GoCD forgets about any commits that were deleted from the library's repo. I don't have nearly a good enough grasp of the database's structure to know how to do this, or even if it is realistically feasible.

In order to prevent this from recurring, you may want to rethink the way the build solution is architected. Instead of pulling directly from the 3rd party's git repo as part of your own build, could you not build the 3rd party library in its own Pipeline and then consume the output of that in other Pipelines? Alternately, if the nature of the code and the libraries is such that they must both be compiled from source at the same time, it may be worth creating your own fork of the 3rd party library on github and then building from that instead of directly from the 3rd party's repo. You would then be responsible for syncing the 3rd party's updates into your fork, but you would also have (almost) full control over your own source, which helps avoid this problem of "Someone on the internet did something and now my build(s) is (are) broken."

Hope this helps,

Jason

[Bojan S.]

Jason,

thanks for your view into GoCD.

Tweaking anything is issue if pipelines are set correctly. Rethinking to adapt to system and its issues is just workaround. That is something that should be avoided.

I will report this as gocd issue on their github page, since it is real problem and although it might feel as borderline case, it is not first time it is encountered.

Thank again for your view into things.

I will leave this question open in case someone from GoCD team stumbles upon it and gives an inside info.

Kind regards,

B