strange pushes on GitHub

80,520 views
Skip to first unread message

domi

unread,
Nov 10, 2013, 10:47:04 AM11/10/13
to Jenkins Developers
Hi everyone and specially Luca,

yesterday something strange has happen to many Jenkins repositories on GitHub (more then 50 Repos)…
Luca Milanesio seems have pushed to many many repositories without really changing anything - at least I have not seen anything changed.
If you look at the activity stream/feed at https://github.com/organizations/jenkinsci and go back to yesterday, you’ll find many pushes to many different repositories.
Maybe I’m wrong and someone has an easy explanation for this?
But for me this looks like someone did not really now how Git/GitHub works and accidentally pushed to every single repo - maybe someone else could also have a look and see if there is something broken now?

regards Domi

Marcelo

unread,
Nov 10, 2013, 10:56:15 AM11/10/13
to jenkin...@googlegroups.com
Regards

Marcelo Rebasti



--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Vincent Latombe

unread,
Nov 10, 2013, 11:21:00 AM11/10/13
to Jenkins Dev
It seems like a lot of repositories are back to previous revisions. git-client-plugin's master now points to 1.4.3-SNAPSHOT whereas it should be pointing to 1.4.7-SNAPSHOT...

Vincent


2013/11/10 Marcelo <mreb...@gmail.com>

ogondza

unread,
Nov 10, 2013, 11:25:25 AM11/10/13
to jenkin...@googlegroups.com
Definitely related, selenium-tests repo lost 17+ commits after such push.

domi

unread,
Nov 10, 2013, 11:51:26 AM11/10/13
to Jenkins Developers, Lu...@milanesio.org
As I don’t know if Luca is following this list too, I’m adding Lucas mail on the to list… (got the email address from his GitHub account)
Please Luca can you explain what you have done to all this repositories?
regards Domi


On 10.11.2013, at 17:25, ogondza <ogo...@gmail.com> wrote:

Definitely related, selenium-tests repo lost 17+ commits after such push.

Arnaud Héritier

unread,
Nov 10, 2013, 1:24:32 PM11/10/13
to jenkin...@googlegroups.com, Jenkins Developers, lu...@milanesio.org
I confirm we lost many commits. damned github that doesn't allow to disable the history rewrite at least on master branch. 
Last commit on Xcode repo is referencing one done by Jerome Lacoste 5 montages ago while KK worked on it recently. 
I don't know how we can restore everything easily. We need to find the recent hash and relative history in a clone we the push of yesterday was retrieved. 


Sent from Mailbox for iPhone

Arnaud Héritier

unread,
Nov 10, 2013, 1:45:03 PM11/10/13
to jenkin...@googlegroups.com, Jenkins Developers, lu...@milanesio.org
S/montages/monthes
S/we/with

Stupid dictionary 


Sent from Mailbox for iPhone


Luca Milanesio

unread,
Nov 10, 2013, 1:55:08 PM11/10/13
to jenkin...@googlegroups.com, Arnaud Héritier
Hi all,
I have triggered an involuntary "forced push" last night on the list of Jenkins-CI plugins indicated below in this e-mail.

My apology 

I did not realise that I actually had forced push permissions and I do apologise for the inconvenience caused.
The operations pushed back the all the branches to around 1 month. The history is not lost and is still on the GitHub server but on detached branches.

The solution

I can raise a request to GitHub to provide the "reflog" of those repositories and restore the branches to the point before my forced push.
Alternatively the owners of those repositories can still perform a "forced push" to restore the correct position of the branches.
(if you would like to do so, please write to the mailing list so that we do not overlap the recovery operations)

The full list

See below the full list of repositories impacted:

antexec-plugin.git
artifactory-plugin.git
associated-files-plugin.git
audit2db-plugin.git
audit-trail-plugin.git
backend-pull-request-greeter.git
beaker-builder-plugin.git
branch-api-plugin.git
build-flow-plugin.git
buildgraph-view.git
build-pipeline-plugin.git
build-timeout-plugin.git
buildtriggerbadge-plugin.git
bytecode-compatibility-transformer.git
ci-game-plugin.git
clearcase-plugin.git
clearcase-ucm-plugin.git
cloudbees-folder-plugin.git
cloudbees-plugin-gateway.git
cloudtest-plugin.git
clover-plugin.git
cobertura-plugin.git
collabnet-plugin.git
collapsing-console-sections-plugin.git
compact-columns-plugin.git
compress-artifacts-plugin.git
conditional-buildstep-plugin.git
config-file-provider-plugin.git
configurationslicing-plugin.git
copyartifact-plugin.git
copy-project-link-plugin.git
copy-to-slave-plugin.git
cppcheck-plugin.git
credentials-plugin.git
crowd2-plugin.git
crowd-plugin.git
customtools-plugin.git
cvsclient.git
cvs-plugin.git
dashboard-view-plugin.git
datical-db-plugin.git
dependency-check-plugin.git
deploy-plugin.git
disable-failed-job-plugin.git
disk-usage-plugin.git
doclinks-plugin.git
dry-plugin.git
dynamic-axis-plugin.git
ec2-plugin.git
elastic-axis-plugin.git
email-ext-plugin.git
envinject-lib.git
envinject-plugin.git
extended-choice-parameter-plugin.git
extra-columns-plugin.git
extras-executable-war.git
extreme-feedback-plugin.git
gearman-plugin.git
gerrit-trigger-plugin.git
gitbucket-plugin.git
git-chooser-alternative-plugin.git
git-client-plugin.git
git-plugin.git
global-build-stats-plugin.git
global-variable-string-parameter-plugin.git
gradle-jpi-plugin.git
grails-plugin.git
greenballs-plugin.git
groovy-postbuild-plugin.git
heavy-job-plugin.git
hockeyapp-plugin.git
http-request-plugin.git
humbug-plugin.git
instant-messaging-plugin.git
integrity-plugin.git
ironmq-notifier-plugin.git
ivytrigger-plugin.git
jacoco-plugin.git
jclouds-plugin.git
jira-plugin.git
jobConfigHistory-plugin.git
job-dsl-plugin.git
job-import-plugin.git
job-poll-action-plugin.git
jquery-plugin.git
jquery-ui-plugin.git
json-lib.git
kiuwan-plugin.git
label-verifier-plugin.git
ldap-plugin.git
leiningen-plugin.git
lib-annotation-indexer.git
lib-task-reactor.git
lib-windows-remote-command.git
literate-cli.git
logfilesizechecker-plugin.git
m2release-plugin.git
m2-repo-reaper-plugin.git
mailer-plugin.git
matrix-auth-plugin.git
maven-hpi-plugin.git
maven-info-plugin.git
mercurial-plugin.git
mesos-plugin.git
metadata-plugin.git
mock-security-realm-plugin.git
msbuild-plugin.git
naginator-plugin.git
nerrvana-plugin.git
nested-view-plugin.git
next-build-number-plugin.git
next-executions-plugin.git
parameterized-trigger-plugin.git
perforce-plugin.git
performance-plugin.git
persona-plugin.git
pitmutation-plugin.git
plain-credentials-plugin.git
plugin-compat-tester.git
postbuildscript-plugin.git
promoted-builds-plugin.git
prqa-plugin.git
publish-over-cifs-plugin.git
puppet-jenkins.git
radiatorview-plugin.git
rapiddeploy-plugin.git
release-plugin.git
repo-plugin.git
rich-text-publisher-plugin.git
robot-plugin.git
run-condition-plugin.git
rvm-plugin.git
scm2job-plugin.git
scm-api-plugin.git
scoring-load-balancer-plugin.git
script-scm-plugin.git
selenium-axis-plugin.git
selenium-builder-plugin.git
selenium-tests.git
skype-im-plugin.git
skytap-cloud-plugin.git
smartfrog-plugin.git
sms-plugin.git
sounds-plugin.git
ssh-agent-plugin.git
ssh-credentials-plugin.git
sshd-module.git
ssh-slaves-plugin.git
starteam-plugin.git
stashnotifier-plugin.git
subversion-plugin.git
suppress-stack-trace-plugin.git
swarm-plugin.git
synergy_scm-plugin.git
tap-plugin.git
teamconcert-plugin.git
testlink-plugin.git
tfs-plugin.git
thin-backup-plugin.git
throttle-concurrent-builds-plugin.git
tikal-multijob-plugin.git
timestamper-plugin.git
token-macro-plugin.git
transifex-plugin.git
translation-plugin.git
trilead-ssh2.git
unity3d-plugin.git
veracode-scanner-plugin.git
view-job-filters-plugin.git
virtualbox-plugin.git
vsphere-cloud-plugin.git
vstestrunner-plugin.git
walldisplay-plugin.git
warnings-plugin.git
weblogic-deployer-plugin.git
winstone.git
wix-plugin.git
ws-cleanup-plugin.git
xcode-plugin.git
xstream.git
xtrigger-lib.git
xunit-plugin.git
xvfb-plugin.git
xvnc-plugin.git

The prevention

Can we prevent this to happen again ?
I personally do not work on any of those repositories but still have "forced push" permissions ... and so many other people have.
I don't see the value of having such power of "potential disruption" associated to my account :-( ... can we remove the forced push by default and enable on a case-by-case basis ?

--- * ---

If you would like to propose an alternative approach to resolve the problem, feel free to follow-up !

... and again ... accept my sincere apologies :-(

Luca.

ogondza

unread,
Nov 10, 2013, 2:05:39 PM11/10/13
to jenkin...@googlegroups.com, Arnaud Héritier
I have already asked Github support to provide hashes of former master positions and referenced this thread. Will report ASAP.

Luca Milanesio

unread,
Nov 10, 2013, 2:10:24 PM11/10/13
to jenkin...@googlegroups.com, Arnaud Héritier, sup...@github.com
*UPDATE*
I have just send the request to GitHub support on-line, looping them as well in this e-mail.

@GitHub Support: please consider this request as a matter of urgency, thank you.

Luca.

Arnaud Héritier

unread,
Nov 10, 2013, 2:30:24 PM11/10/13
to ogondza, jenkin...@googlegroups.com
Thx guys for these details. We are all humans and subject of doing errors. 
I hope we'll have some help from github but I'm not sure they may spend many time to help us for an error not coming from their side. 
In any case it is recommended to NOT PULL current master branches from these repositories, your local copy may be more up-to-date. If you have a recent hash you can at least try to push them in github under a branch named "master-fix" the time we find the best solution to repare all master repositories. 
If you commited on these repositories since yesterday verify that you didn't started your work from the wrong old hash. 

Idea : in our CI server logs we should be able to find the lastest hash of each repo in the related plugin build. After that we have to find how to restore them. It's easy to do it when you have this commit locally but AFAIK it's not if it is only in the remote (you don't get orphelin commits when you clone a repo)

Arnaud


Sent from Mailbox for iPhone


Luca Milanesio

unread,
Nov 10, 2013, 2:41:40 PM11/10/13
to jenkin...@googlegroups.com, Arnaud Héritier, sup...@github.com
*UPDATE* after a cross-check I need to add 2 extra repositories to the list:
accurev-plugin
analysis-core-plugin

The total number of impacted repositories should then be: 186

@ALL: is there anyone with direct contact of some GitHub-er ? It would be best to resolve the problem *before* the start of the working week on Monday.

Thanks to everyone that contributed to resolve the problem !

Luca.

Luca Milanesio

unread,
Nov 10, 2013, 2:47:50 PM11/10/13
to jenkin...@googlegroups.com, ogondza
Hi Arnaud,
thanks for the understanding :-) I am used to warn other people from doing any "forced push" ... and Murphy's law wanted this time happening to me :-(

On 10 Nov 2013, at 19:30, Arnaud Héritier <aher...@gmail.com> wrote:

> Thx guys for these details. We are all humans and subject of doing errors.
> I hope we'll have some help from github but I'm not sure they may spend many time to help us for an error not coming from their side.

Technically yes, but for the "customer relationship" and because it is an easy recovery (git reflog + git branch), they could make some effort to help us out.
JenkinsCI is possibly the largest customer they have worldwide ;-)

> In any case it is recommended to NOT PULL current master branches from these repositories, your local copy may be more up-to-date. If you have a recent hash you can at least try to push them in github under a branch named "master-fix" the time we find the best solution to repare all master repositories.
> If you commited on these repositories since yesterday verify that you didn't started your work from the wrong old hash.
>
> Idea : in our CI server logs we should be able to find the lastest hash of each repo in the related plugin build. After that we have to find how to restore them. It's easy to do it when you have this commit locally but AFAIK it's not if it is only in the remote (you don't get orphelin commits when you clone a repo)

Yes, that's what I thought as well ... but definitely the "git reflog" on GitHub server would be more reliable.
The CI workspace (assuming that Git clones are not shallow copies) may have as well those commits as well, but again GitHub would be safer and better IMHO.

Luca

Marcus Bauer

unread,
Nov 10, 2013, 2:52:39 PM11/10/13
to jenkin...@googlegroups.com
Hi,

I don't think GitHub.com has any possibilities for disabling force pushes, this seems to be exclusive to GH Enterprise only.

The JaCoCo repo where I initially noticed this was restored by Dominik Stadler (centic9) few minutes ago.

Marcus

Mirko Friedenhagen

unread,
Nov 10, 2013, 3:00:30 PM11/10/13
to jenkin...@googlegroups.com
Hello,

I saw these strange pushes as well, jobConfigHistory-plugin was
failing because master was reset to a commit which was two months old.
I see that https://github.com/jenkinsci/maven-hpi-plugin/tree/master
is pointing to an old commit as well, the new version 1.106 is much
newer than master.

Regards
Mirko
Regards Mirko
--
http://illegalstateexception.blogspot.com/
https://github.com/mfriedenhagen/
https://bitbucket.org/mfriedenhagen/

Luca Milanesio

unread,
Nov 10, 2013, 4:40:28 PM11/10/13
to jenkin...@googlegroups.com
That's really pitty :-( ... force push are dangerous, especially if you don't have control over the Git Server.

Typically recovering a force push is straightforward:
1. git reflog > look at the SHA-1 before the forced push
2. git branch -f <name> <sha-1>

But if you don't have control over the Git repo on the Server, that you need to prevent force push to happen: unless they want everyone to buy GH Enterprise !

I still hope GitHub will do 1. and 2. for us :-)

Luca.

Dominik Stadler

unread,
Nov 10, 2013, 4:59:42 PM11/10/13
to jenkin...@googlegroups.com
Hi,

I didn't know that this was a bigger problem and thus just pushed the previous commits again, in case of the jacoco-plugin repo it was easy as I just committed changes yesterday and thus could easily push the latest changes together with the missing commits.

The really bad thing in my view is that as a user of the repo you do not see at all that it happened. Git and Github is all about history of changes, but in this case your repo looks like a month ago with no visible trace on github that something newer was there at some time... Quite dangerous....

Dominik.

Slide

unread,
Nov 10, 2013, 5:06:36 PM11/10/13
to Jenkins Dev, Arnaud Héritier
I did a force push to email-ext and it seems to be fine now.

Thanks,

slide

Arnaud Héritier

unread,
Nov 10, 2013, 5:11:34 PM11/10/13
to Slide, Jenkins Dev
In theory, to restore the master to the latest hash (and others branches - As I can see in Github organisation history) no --force will be required. You won't change the history, you just re-add missing commits on top of the old one.
--
-----
Arnaud Héritier
Mail/GTalk: aheritier AT gmail DOT com
Twitter/Skype : aheritier

Arnaud Héritier

unread,
Nov 10, 2013, 5:16:55 PM11/10/13
to jenkin...@googlegroups.com
I hope that Github will help us. That may save us a lot of time. Perhaps giving us a copy of all repositories could be enough and we may dig ourselves in reflogs to restore branches.
Yes that's really annoying that they don't allow us to block force push on some branches. Because of that when I migrated my company from subversion to Github I setup a daily external clone/backup process of all repositories in a case where we had such issue and we weren't easily able to identify a recent clone amonst ours developers (which is a children game in a company compared in an OSS like Jenkins)

Christopher Orr

unread,
Nov 10, 2013, 5:42:53 PM11/10/13
to jenkin...@googlegroups.com
With 678 members in the jenkinsci organisation, we're probably
relatively exceptional; I can't imagine GitHub have much incentive to
implement the type of fine-grained force-push permissions we would need.

Plus I think even with the number of members we have, git accidents are
very rare, i.e. it's likely easier to fix mistakes in git, than to
maintain a big set of permissions for new and existing users/repos.

BTW, out of curiosity, how did you manage to force push such a large
number of repositories at the same time? :)

Regards,
Chris


On 11/10/2013 10:40 PM, Luca Milanesio wrote:
> That's really pitty :-( ... force push are dangerous, especially if you
> don't have control over the Git Server.
>
> Typically recovering a force push is straightforward:
> 1. git reflog > look at the SHA-1 before the forced push
> 2. git branch -f <name> <sha-1>
>
> But if you don't have control over the Git repo on the Server, that you
> need to prevent force push to happen: unless they want everyone to buy
> GH Enterprise !
>
> I still hope GitHub will do 1. and 2. for us :-)
>
> Luca.
>
> On 10 Nov 2013, at 19:52, Marcus Bauer <mab...@gmail.com
> <mailto:mab...@gmail.com>> wrote:
>
>> Hi,
>>
>> I don't think GitHub.com <http://GitHub.com> has any possibilities for
>> disabling force pushes, this seems to be exclusive to GH Enterprise
>> <https://enterprise.github.com/help/articles/disable-force-pushes> only.
>>
>> The JaCoCo repo where I initially noticed this was restored by Dominik
>> Stadler (centic9) few minutes ago.
>>
>> Marcus
>>
>> Am Sonntag, 10. November 2013 19:55:08 UTC+1 schrieb lucamilanesio:
>>
>> Hi all,
>> I have triggered an involuntary "forced push" last night on the
>> list of Jenkins-CI plugins indicated below in this e-mail.
>>
>> *_My apology _*
>>
>> I did not realise that I actually had forced push permissions and
>> I do apologise for the inconvenience caused.
>> The operations pushed back the all the branches to around 1 month.
>> The history is not lost and is still on the GitHub server but on
>> detached branches.
>>
>> *_The solution_*
>> *_
>> _*
>> I can raise a request to GitHub to provide the "reflog" of those
>> repositories and restore the branches to the point before my
>> forced push.
>> /_Alternatively the owners of those repositories can still perform
>> a "forced push" to restore the correct position of the branches._/
>> (if you would like to do so, *_please write to the mailing list so
>> that we do not overlap the recovery operations_*)
>>
>> *_The full list_*
>> *_The prevention_*

Luca Milanesio

unread,
Nov 10, 2013, 6:24:09 PM11/10/13
to jenkin...@googlegroups.com
GitHub model is based on pull requests and not on shared Git repositories: that's why protections against forced push is typically not needed (typically repo is pushed by only its maintainers and not by contributors).
The way we use GitHub with Jenkins is not very typical for  them :-)

On 10 Nov 2013, at 22:42, Christopher Orr <ch...@orr.me.uk> wrote:

With 678 members in the jenkinsci organisation, we're probably relatively exceptional; I can't imagine GitHub have much incentive to implement the type of fine-grained force-push permissions we would need.

Plus I think even with the number of members we have, git accidents are very rare, i.e. it's likely easier to fix mistakes in git, than to maintain a big set of permissions for new and existing users/repos.

Yes they are rare ! ... but when they happen you need to sort them out :-(

BTW, out of curiosity, how did you manage to force push such a large number of repositories at the same time? :)

Problems with the Gerrit replication plugin ... unfortunate combination of three events (Murphy's law ?):
- working on a directory where I had the Jenkins plugins repos cloned (but outdated to around 1 month ago)
- having a Gerrit replication plugin pointing to that directory (I did not realised was using *forced push* that until later today)
- having the permissions of making *forced push* on repos that I did not work on (I did not realised that until today)

I have removed those local repos on my machine ... so can't do it again, but we need to sort out the mess in the meantime :-(
I am submitting now a patch to Gerrit replication plugin to *forbid* forced push by configuration: so at least I can block them from my side.

Possibly a bit late ... closing the gates when the sheep just escaped ;-(

Luca.

Christopher Orr

unread,
Nov 10, 2013, 6:42:46 PM11/10/13
to jenkin...@googlegroups.com
On 11/11/2013 12:24 AM, Luca Milanesio wrote:
>> BTW, out of curiosity, how did you manage to force push such a large
>> number of repositories at the same time? :)
>
> Problems with the Gerrit replication plugin ... unfortunate combination
> of three events (Murphy's law ?):
> - working on a directory where I had the Jenkins plugins repos cloned
> (but outdated to around 1 month ago)
> - having a Gerrit replication plugin pointing to that directory (I did
> not realised was using *forced push* that until later today)
> - having the permissions of making *forced push* on repos that I did not
> work on (I did not realised that until today)
>
> I have removed those local repos on my machine ... so can't do it again,
> but we need to sort out the mess in the meantime :-(
> I am submitting now a patch to Gerrit replication plugin to **forbid*
> forced push by configuration*: so at least I can block them from my side.

Yeah, that's unlucky.

One thing I sometimes do for repos where I have commit access, but don't
want to push, is to clone the repository via HTTP. That way, any
accidental attempts at pushing won't use my ssh key, and so GitHub will
prompt me for my password.

Regards,
Chris

Luca Milanesio

unread,
Nov 11, 2013, 3:11:17 AM11/11/13
to jenkin...@googlegroups.com
Does anyone have news from GitHub ?
They haven't responded at all to my customer support request :-(

Luca.

Vojtech Juranek

unread,
Nov 11, 2013, 3:15:03 AM11/11/13
to jenkin...@googlegroups.com
On Sunday 10 November 2013 21:40:28 Luca Milanesio wrote:
> That's really pitty :-( ... force push are dangerous, especially if you
> don't have control over the Git Server.

I wonder if we can use our all.git [1] somehow (in the worst case scenario
that github doesn't help us). When it try to clone it, it fails with remote
error and when looking into web UI the changes are already synchronized with
github. But IMHO still worth to investigate, orphan commits could still be
there

[1] http://git.jenkins-ci.org/?p=all.git;a=summary
signature.asc

Emanuele Zattin

unread,
Nov 11, 2013, 3:21:31 AM11/11/13
to jenkin...@googlegroups.com
Maybe somebody from the bay area could try driving to the Github office on Monday morning. Scott Chacon would probably be good bet.

Emanuele Zattin
---------------------------------------------------
-I don't have to know an answer. I don't feel frightened by not knowing things; by being lost in a mysterious universe without any purpose — which is the way it really is, as far as I can tell, possibly. It doesn't frighten me.- Richard Feynman

nicolas de loof

unread,
Nov 11, 2013, 4:05:30 AM11/11/13
to jenkin...@googlegroups.com, Arnaud Héritier
git fetch + git push restored git-client and git plugins history


2013/11/10 Luca Milanesio <luca.mi...@gmail.com>

Stephen Connolly

unread,
Nov 11, 2013, 4:09:04 AM11/11/13
to jenkin...@googlegroups.com, Arnaud Héritier
I had pushed releases of credentials, ssh-credentials, ssh-agent and ec2 on Friday, so I just re-pushed them. I don't think anyone made changes to scm-api or branch-api since my last push, so I pushed those from my local also

Mads Nielsen

unread,
Nov 11, 2013, 5:06:43 AM11/11/13
to jenkin...@googlegroups.com, Arnaud Héritier
What is the recommended way to get a full restore, in the case where you do not happen to have a local clone of the repository which was cloned before the accident?

Mads Nielsen
Consultant
Praqma A/S

Tel: +45 50 98 18 09
Mail: m...@praqma.net
web: www.praqma.net

nicolas de loof

unread,
Nov 11, 2013, 5:25:17 AM11/11/13
to jenkin...@googlegroups.com, Arnaud Héritier
in worst case, we have a clone on jenkins.ci.cloudbees.com we can use to push back to github.
just need some scripting to use it.

would you like me to look into this option ?


2013/11/11 Mads Nielsen <m...@praqma.net>

Arnaud Héritier

unread,
Nov 11, 2013, 5:36:17 AM11/11/13
to nicolas de loof, jenkin...@googlegroups.com
Hi Nicolas. 

  For me it would be the safer issue. You could wait for tomorrow (it's a day off today in France) but we just need to take care to not loose these repositories. 

Sent from Mailbox for iPhone


nicolas de loof

unread,
Nov 11, 2013, 5:38:50 AM11/11/13
to Arnaud Héritier, jenkin...@googlegroups.com
I can look into this while traveling to devoxx tomorrow


2013/11/11 Arnaud Héritier <aher...@gmail.com>

Luca Milanesio

unread,
Nov 11, 2013, 5:58:04 AM11/11/13
to jenkin...@googlegroups.com, nicolas de loof, Arnaud Héritier
Yes, I think that would be a sensible approach.
As some of the people started already to push their up-to-date repo, it is better a push from the CI node rather than a "reset" from the GitHub guys at this stage.

Luca.

Hakan Tandoğan

unread,
Nov 11, 2013, 7:16:43 AM11/11/13
to jenkin...@googlegroups.com, Arnaud Héritier
I pushed transifex-plugin.git from my local tree, all fine for this plugin now.

Regards,
Hakan

Mike Caspar

unread,
Nov 11, 2013, 7:22:58 AM11/11/13
to jenkin...@googlegroups.com
HI there,
I just tried to connect to check out the plugin I maintain (ironmq-notifier).
I have lost access to the repo completely.
I suspect that since I'm new, my rights were reverted due to this somehow.
Could someone add me back into the plugin ironmq-notifier.
Thanks

userid:mikecaspar

Mike Caspar

unread,
Nov 11, 2013, 7:24:59 AM11/11/13
to jenkin...@googlegroups.com
Actually, looks like it was just some weird caching problem.
Back into the repo now.

nicolas de loof

unread,
Nov 11, 2013, 9:09:29 AM11/11/13
to jenkin...@googlegroups.com, Arnaud Héritier
I've pushed plugin's master branches from jenkins.ci.cloudbees.com

some of then were rejected, like 

g...@github.com:jenkinsci/warnings-plugin.git

 ! [rejected]        master -> master (non-fast-forward)


Anyway I guess most plugin repo should be restored now.


2013/11/11 Hakan Tandoğan <hakan.t...@gmail.com>

Mads Nielsen

unread,
Nov 11, 2013, 9:16:32 AM11/11/13
to jenkin...@googlegroups.com, Arnaud Héritier
I can confirm that the ones i was interested in are OK: 


Thanks nicolas!

Vivekanand S V

unread,
Nov 11, 2013, 9:20:06 AM11/11/13
to jenkin...@googlegroups.com
This one https://github.com/jenkinsci/copy-to-slave-plugin recovered only some commits.

nicolas de loof

unread,
Nov 11, 2013, 9:26:14 AM11/11/13
to jenkin...@googlegroups.com
master branch is up-to-date with the one on jenkins.ci.cloudbees.com

do you have a local clone you could push from ?


2013/11/11 Vivekanand S V <svvive...@gmail.com>

oliver gondža

unread,
Nov 11, 2013, 9:28:18 AM11/11/13
to jenkin...@googlegroups.com, nicolas de loof
Thanks nicolas,

Could you list the repos that rejected the push and point us to the backup
repos so we can fix the rest manually? For instance
next-build-number-plugin is missing 6 commits compared to my local copy. I
assume this is not the only one.

--
oliver

Ulli Hafner

unread,
Nov 11, 2013, 9:33:52 AM11/11/13
to jenkin...@googlegroups.com
I already pushed my local commits again, so the history of the warnings plug-in is up to date again.
signature.asc

nicolas de loof

unread,
Nov 11, 2013, 9:37:36 AM11/11/13
to oliver gondža, jenkin...@googlegroups.com

Here are the repo that fail to git push as "[rejected]        master -> master (non-fast-forward)"


branch-api-plugin.git

credentials-plugin.git

deployit-plugin.git

ec2-cloud-axis-plugin.git

ec2-plugin.git

email-ext-plugin.git

flexible-publish-plugin.git

gerrit-trigger-plugin.git

git-client-plugin.git

git-plugin.git

ironmq-notifier-plugin.git

jacoco-plugin.git

jobConfigHistory-plugin.git

literate-plugin.git

maven-info-plugin.git

maven-plugin.git

perforce-plugin.git

rebuild-plugin.git

sounds-plugin.git

ssh-slaves-plugin.git

subversion-plugin.git

synergy_scm-plugin.git

testlink-plugin.git

tfs-plugin.git

transifex-plugin.git

warnings-plugin.git












2013/11/11 oliver gondža <ogo...@gmail.com>

Slide

unread,
Nov 11, 2013, 9:40:35 AM11/11/13
to Jenkins Dev, oliver gondža
I already pushed the correct stuff to email-ext-plugin.git, so that one is done.

Regards,

slide


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Iñaki Albors

unread,
Nov 11, 2013, 10:09:37 AM11/11/13
to jenkinsci-dev
next-executions-plugin restored

Vivekanand S V

unread,
Nov 11, 2013, 10:16:44 AM11/11/13
to jenkin...@googlegroups.com
@nicolas, I have a local clone which I last used to push a release. What do I need to do ?

Stefan Brausch

unread,
Nov 11, 2013, 10:32:33 AM11/11/13
to jenkin...@googlegroups.com
logfilesizechecker and scm2job restored from local copy  (wasn’t complete)

nicolas de loof

unread,
Nov 11, 2013, 10:56:33 AM11/11/13
to jenkin...@googlegroups.com
just git push origin master

the unintentional force push just reverted history some weeks ago, so this will re-add the missing commits


2013/11/11 Vivekanand S V <svvive...@gmail.com>
@nicolas, I have a local clone which I last used to push a release. What do I need to do ?

Vivekanand S V

unread,
Nov 11, 2013, 11:09:12 AM11/11/13
to jenkin...@googlegroups.com
@nicolas, its done, copy-to-slave is upto date now.


Thanks a lot.

Vojtech Juranek

unread,
Nov 11, 2013, 2:04:49 PM11/11/13
to jenkin...@googlegroups.com
Hi,
just FIY, if your plugin is not on this list, this doesn't mean the commits
were successfully restored - e.g. one of my plugins (beaker-builder) is not on
the list and the last commit is 3 months old [1], while actual last commit is
one week old [2], so please double check your plugins
Vojta

[1] https://github.com/jenkinsci/beaker-builder-plugin/commits/master
[2] https://github.com/vjuranek/beaker-builder-plugin/commits/master
> 2013/11/11 oliver gondža <ogo...@gmail.com>
signature.asc

nicolas de loof

unread,
Nov 11, 2013, 2:17:30 PM11/11/13
to jenkin...@googlegroups.com
yes indeed, this is just the last state on CI job, no guarantee it hasn't been updated in the meantime based on github forced push


2013/11/11 Vojtech Juranek <vjur...@redhat.com>

Mirko Friedenhagen

unread,
Nov 11, 2013, 3:00:53 PM11/11/13
to jenkin...@googlegroups.com
I pushed my local copy of jobConfigHistory-plugin without force and
all commits are current now.
Regards Mirko
--
http://illegalstateexception.blogspot.com/
https://github.com/mfriedenhagen/
https://bitbucket.org/mfriedenhagen/

Rob Petti

unread,
Nov 11, 2013, 3:09:24 PM11/11/13
to jenkin...@googlegroups.com, Arnaud Héritier
perforce-plugin.git has been restored from my local copy.

It may be beneficial to look into implementing finer grained access control for individual repos. For example, I thought that only myself, Oleg, and the Jenkins project owners had write access to the perforce plugin. If everyone has access to everything, then this sort of thing could happen again.

On Sunday, 10 November 2013 11:55:08 UTC-7, lucamilanesio wrote:
Hi all,
I have triggered an involuntary "forced push" last night on the list of Jenkins-CI plugins indicated below in this e-mail.

My apology 

I did not realise that I actually had forced push permissions and I do apologise for the inconvenience caused.
The operations pushed back the all the branches to around 1 month. The history is not lost and is still on the GitHub server but on detached branches.

The solution

I can raise a request to GitHub to provide the "reflog" of those repositories and restore the branches to the point before my forced push.
Alternatively the owners of those repositories can still perform a "forced push" to restore the correct position of the branches.
(if you would like to do so, please write to the mailing list so that we do not overlap the recovery operations)

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Kohsuke Kawaguchi

unread,
Nov 11, 2013, 3:19:43 PM11/11/13
to jenkin...@googlegroups.com
On 11/10/2013 10:55 AM, Luca Milanesio wrote:
> Hi all,
> I have triggered an involuntary "forced push" last night on the list of
> Jenkins-CI plugins indicated below in this e-mail.
>
> *_My apology _*
>
> I did not realise that I actually had forced push permissions and I do
> apologise for the inconvenience caused.
> The operations pushed back the all the branches to around 1 month. The
> history is not lost and is still on the GitHub server but on detached
> branches.
>
> *_The solution_*
> *_
> _*
> I can raise a request to GitHub to provide the "reflog" of those
> repositories and restore the branches to the point before my forced push.
> /_Alternatively the owners of those repositories can still perform a
> "forced push" to restore the correct position of the branches._/
> (if you would like to do so, *_please write to the mailing list so that
> we do not overlap the recovery operations_*)

If they can just create refs for all the tips of the affected branches
before a forced push in a distinct names, then we can write a script
that compares that with the current tip of the branches (which may or
may not be fixed by someone manually already) and do the right thing.



>
> *_The full list_*
> *_The prevention_*
>
> Can we prevent this to happen again ?
> I personally do not work on any of those repositories but still have
> "forced push" permissions ... and so many other people have.
> I don't see the value of having such power of "potential disruption"
> associated to my account :-( ... can we remove the forced push by
> default and enable on a case-by-case basis ?
>
> --- * ---
>
> If you would like to propose an alternative approach to resolve the
> problem, feel free to follow-up !
>
> ... and again ... accept my sincere apologies :-(
>
> Luca.
>
> On 10 Nov 2013, at 18:24, Arnaud H�ritier <aher...@gmail.com
> <mailto:aher...@gmail.com>> wrote:
>
>> I confirm we lost many commits. damned github that doesn't allow to
>> disable the history rewrite at least on master branch.
>> Last commit on Xcode repo is referencing one done by Jerome Lacoste 5
>> montages ago while KK worked on it recently.
>> I don't know how we can restore everything easily. We need to find the
>> recent hash and relative history in a clone we the push of yesterday
>> was retrieved.
>>
>> �
>> Sent from Mailbox <https://www.dropbox.com/mailbox> for iPhone
>>
>>
>> On Sun, Nov 10, 2013 at 5:51 PM, domi <do...@fortysix.ch
>> <mailto:do...@fortysix.ch>> wrote:
>>
>> As I don�t know if Luca is following this list too, I�m adding
>> Lucas mail on the to list� (got the email address from his GitHub
>> account)
>> Please Luca can you explain what you have done to all this
>> repositories?
>> regards Domi
>>
>>
>> On 10.11.2013, at 17:25, ogondza <ogo...@gmail.com
>> <mailto:ogo...@gmail.com>> wrote:
>>
>>> Definitely related, selenium-tests repo lost 17+ commits after
>>> such push.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the
>>> Google Groups "Jenkins Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to jenkinsci-de...@googlegroups.com
>>> <mailto:jenkinsci-de...@googlegroups.com>.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to jenkinsci-de...@googlegroups.com
>> <mailto:jenkinsci-de...@googlegroups.com>.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkinsci-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


--
Kohsuke Kawaguchi http://kohsuke.org/

Kohsuke Kawaguchi

unread,
Nov 11, 2013, 3:23:24 PM11/11/13
to jenkin...@googlegroups.com

Yes, I was thinking about the same. The last run of this is Nov 9,
11:24pm in EST.

I really hope this is before the incident. But I'll find out soon.

On 11/11/2013 12:15 AM, Vojtech Juranek wrote:
> On Sunday 10 November 2013 21:40:28 Luca Milanesio wrote:
>> That's really pitty :-( ... force push are dangerous, especially if you
>> don't have control over the Git Server.
>
> I wonder if we can use our all.git [1] somehow (in the worst case scenario
> that github doesn't help us). When it try to clone it, it fails with remote
> error and when looking into web UI the changes are already synchronized with
> github. But IMHO still worth to investigate, orphan commits could still be
> there
>
> [1] http://git.jenkins-ci.org/?p=all.git;a=summary
>


--
Kohsuke Kawaguchi | CloudBees, Inc. | http://cloudbees.com/
Try Jenkins Enterprise, our professional version of Jenkins

Andrew Bayer

unread,
Nov 11, 2013, 3:25:57 PM11/11/13
to jenkin...@googlegroups.com
fwiw, gradle-jpi-plugin, throttle-concurrent-builds-plugin, jclouds-plugin and associated-files-plugin are all up to date now.


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

Kohsuke Kawaguchi

unread,
Nov 11, 2013, 4:22:29 PM11/11/13
to jenkin...@googlegroups.com, oliver gondža

You should push 'master' in these repos to a separate ref so that we can
compare.
> 2013/11/11 oliver gondža <ogo...@gmail.com <mailto:ogo...@gmail.com>>
>
> Thanks nicolas,
>
> Could you list the repos that rejected the push and point us to the
> backup repos so we can fix the rest manually? For instance
> next-build-number-plugin is missing 6 commits compared to my local
> copy. I assume this is not the only one.
>
> --
> oliver
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkinsci-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


Kohsuke Kawaguchi

unread,
Nov 11, 2013, 5:08:19 PM11/11/13
to jenkin...@googlegroups.com
On 11/11/2013 12:23 PM, Kohsuke Kawaguchi wrote:
>
> Yes, I was thinking about the same. The last run of this is Nov 9,
> 11:24pm in EST.
>
> I really hope this is before the incident. But I'll find out soon.

Unfortunately, it appears that the last sync process has run after
luca's "push -f".

I'll hold on to this repo just in case, but resurrecting lost commits
from this appears hopeless.

Luca Milanesio

unread,
Nov 11, 2013, 5:22:55 PM11/11/13
to jenkin...@googlegroups.com
Good news from GitHub: they have extracted the full list of SHA-1 before the forced push !
Many thanks to Nathan Witmer :-)

See below the full CSV with the SHA-1.
He created as well a branch named 'recovery' that points to the candidate point for restoring the master branch.

Hope this will help to sort out the remaining repos.

Luca.

> Hi Luca.
>
> Oh man, that sinking feeling!
>
> But, no worries: I've gone through each of the repos listed above and done the following:
>
> * retrieved the SHA for the previous `master` before you force-pushed
> * created a branch called `recovery` pointing to each former master
>
> In some cases, these are the same.
>
> I can go further and reset the master refs to their former shas if you'd like, or you can recover these yourself. To do so, in each repo:
>
> $ git fetch
> $ git checkout master
> $ git reset --hard origin/recovery
> $ git push --force origin master
>
> I've attached a csv containing the shas (former master and forced master) for each branch, for your reference.
>
> Good luck!
>
> Nathan.
jenkinsci_restore.csv

Kohsuke Kawaguchi

unread,
Nov 11, 2013, 5:27:14 PM11/11/13
to jenkin...@googlegroups.com
We at the #jenkins channel in IRC were just wondering what the 'recovery' branch is.

I think it makes sense to bring back 'master' to 'recovery' where it is fast-forward, but let's walk bit slowly here...


2013/11/11 Luca Milanesio <luca.mi...@gmail.com>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.




--
Kohsuke Kawaguchi

Kohsuke Kawaguchi

unread,
Nov 11, 2013, 5:46:48 PM11/11/13
to jenkin...@googlegroups.com
On 11/11/2013 02:27 PM, Kohsuke Kawaguchi wrote:
> We at the #jenkins channel in IRC were just wondering what the
> 'recovery' branch is.
>
> I think it makes sense to bring back 'master' to 'recovery' where it is
> fast-forward, but let's walk bit slowly here...

I'm downloading all the affected repositories as is now and. I'd like to
back them up first before attempting any further scripted edits to refs.

Once that's completed, I propose is we run "git push
origin/recovery:master" on each repo. The purpose of this command is:

1. if the current 'master' (which lost some commits) can be
fast-forwarded to 'recovery', we'll do so.
2. if the current 'master' has diverged from the 'recovery',
for example because it has already been pushed or a separate
recovery was attempted, then this git-push will fail.

For repositories where 'master' didn't fast-forward to 'recovery', I'll
create a list of them and we'll look at them individually. Hopefully
they won't be large in numbers.

If anyone else have any thoughts about how to recover all the
repositories en mass, please let us know.


> 2013/11/11 Luca Milanesio <luca.mi...@gmail.com
> <mailto:luca.mi...@gmail.com>>
> <mailto:jenkinsci-dev%2Bunsu...@googlegroups.com>.
> > For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to jenkinsci-de...@googlegroups.com
> <mailto:jenkinsci-dev%2Bunsu...@googlegroups.com>.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
>
> --
> Kohsuke Kawaguchi
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkinsci-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


--

Arnaud Héritier

unread,
Nov 11, 2013, 6:15:22 PM11/11/13
to jenkin...@googlegroups.com
On Mon, Nov 11, 2013 at 11:27 PM, Kohsuke Kawaguchi <k...@kohsuke.org> wrote:
We at the #jenkins channel in IRC were just wondering what the 'recovery' branch is.


A big thx to Nathan Witmer !!

 
I think it makes sense to bring back 'master' to 'recovery' where it is fast-forward, but let's walk bit slowly here...

+1



--
-----
Arnaud Héritier
Mail/GTalk: aheritier AT gmail DOT com
Twitter/Skype : aheritier

Arnaud Héritier

unread,
Nov 11, 2013, 6:31:22 PM11/11/13
to jenkin...@googlegroups.com
On Mon, Nov 11, 2013 at 11:46 PM, Kohsuke Kawaguchi <kkawa...@cloudbees.com> wrote:
On 11/11/2013 02:27 PM, Kohsuke Kawaguchi wrote:
We at the #jenkins channel in IRC were just wondering what the
'recovery' branch is.

I think it makes sense to bring back 'master' to 'recovery' where it is
fast-forward, but let's walk bit slowly here...

I'm downloading all the affected repositories as is now and. I'd like to back them up first before attempting any further scripted edits to refs.

Once that's completed, I propose is we run "git push origin/recovery:master" on each repo. The purpose of this command is:

  1. if the current 'master' (which lost some commits) can be
     fast-forwarded to 'recovery', we'll do so.
  2. if the current 'master' has diverged from the 'recovery',
     for example because it has already been pushed or a separate
     recovery was attempted, then this git-push will fail.

For repositories where 'master' didn't fast-forward to 'recovery', I'll create a list of them and we'll look at them individually. Hopefully they won't be large in numbers.

If anyone else have any thoughts about how to recover all the repositories en mass, please let us know.


Perfect

Nothing better to propose.

Good luck

 


2013/11/11 Luca Milanesio <luca.mi...@gmail.com
<mailto:luca.milanesio@gmail.com>>

    <mailto:jenkinsci-dev%2Bunsu...@googlegroups.com>.
     > For more options, visit https://groups.google.com/groups/opt_out.


    --
    You received this message because you are subscribed to the Google
    Groups "Jenkins Developers" group.
    To unsubscribe from this group and stop receiving emails from it,

    <mailto:jenkinsci-dev%2Bunsu...@googlegroups.com>.
    For more options, visit https://groups.google.com/groups/opt_out.




--
Kohsuke Kawaguchi

--
You received this message because you are subscribed to the Google
Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send

For more options, visit https://groups.google.com/groups/opt_out.


--
Kohsuke Kawaguchi | CloudBees, Inc. | http://cloudbees.com/
Try Jenkins Enterprise, our professional version of Jenkins

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.



--

Kohsuke Kawaguchi

unread,
Nov 11, 2013, 9:13:49 PM11/11/13
to jenkin...@googlegroups.com

Any more sanity checks on this course of action before I actually run
them? I've only heard from Arnaud...

Domi

unread,
Nov 11, 2013, 11:50:35 PM11/11/13
to jenkin...@googlegroups.com
Sounds perfect for me!
Domi

Wes Turner

unread,
Nov 12, 2013, 3:04:21 AM11/12/13
to jenkin...@googlegroups.com

> I can't imagine GitHub have much incentive to 
> implement the type of fine-grained force-push permissions we would need

There are fine-grained "File and branch conditions" in mercurial-server
that can prevent these sorts of pushes from succeeding. [1]

[1] http://dev.lshift.net/paul/mercurial-server/docbook.html#id1333360

On Sunday, November 10, 2013 4:42:53 PM UTC-6, Christopher wrote:
With 678 members in the jenkinsci organisation, we're probably
relatively exceptional; I can't imagine GitHub have much incentive to
implement the type of fine-grained force-push permissions we would need.

Plus I think even with the number of members we have, git accidents are
very rare, i.e. it's likely easier to fix mistakes in git, than to
maintain a big set of permissions for new and existing users/repos.

BTW, out of curiosity, how did you manage to force push such a large
number of repositories at the same time? :)

Regards,
Chris


On 11/10/2013 10:40 PM, Luca Milanesio wrote:
> That's really pitty :-( ... force push are dangerous, especially if you
> don't have control over the Git Server.
>
> Typically recovering a force push is straightforward:
> 1. git reflog > look at the SHA-1 before the forced push
> 2. git branch -f <name> <sha-1>
>
> But if you don't have control over the Git repo on the Server, that you
> need to prevent force push to happen: unless they want everyone to buy
> GH Enterprise !
>
> I still hope GitHub will do 1. and 2. for us :-)
>
> Luca.
>
> On 10 Nov 2013, at 19:52, Marcus Bauer <mab...@gmail.com
> <mailto:mab...@gmail.com>> wrote:
>
>> Hi,
>>
>> I don't think GitHub.com <http://GitHub.com> has any possibilities for
>> disabling force pushes, this seems to be exclusive to GH Enterprise
>> <https://enterprise.github.com/help/articles/disable-force-pushes> only.
>>
>> The JaCoCo repo where I initially noticed this was restored by Dominik
>> Stadler (centic9) few minutes ago.
>>
>> Marcus
>>
>> Am Sonntag, 10. November 2013 19:55:08 UTC+1 schrieb lucamilanesio:
>>
>>     Hi all,
>>     I have triggered an involuntary "forced push" last night on the
>>     list of Jenkins-CI plugins indicated below in this e-mail.
>>
>>     *_My apology _*
>>
>>     I did not realise that I actually had forced push permissions and
>>     I do apologise for the inconvenience caused.
>>     The operations pushed back the all the branches to around 1 month.
>>     The history is not lost and is still on the GitHub server but on
>>     detached branches.
>>
>>     *_The solution_*
>>     *_
>>     _*
>>     I can raise a request to GitHub to provide the "reflog" of those
>>     repositories and restore the branches to the point before my
>>     forced push.
>>     /_Alternatively the owners of those repositories can still perform
>>     a "forced push" to restore the correct position of the branches._/
>>     (if you would like to do so, *_please write to the mailing list so
>>     that we do not overlap the recovery operations_*)
>>

Ikedam

unread,
Nov 12, 2013, 7:16:27 AM11/12/13
to jenkin...@googlegroups.com
Unfortunately, there may be something wrong with this list...

scoring-load-balancer-plugin have same orig_master and forced_master, 634609e.
And the version in its pom.xml  is 1.0.1-SNAPSHOT.

But I have released scoring-load-balancer-1.0.1 last week.
The latest commit should be:
https://github.com/jenkinsci/scoring-load-balancer-plugin/commit/0ca553b933521ba82746a491b3c29d8c6cc2ae89

There may be same problems in other plugins if they are updated in a week.

Luca Milanesio

unread,
Nov 12, 2013, 8:10:50 AM11/12/13
to jenkin...@googlegroups.com
Looking again at the list right now: there are a few in the same situation :-(

Luca
---------
Sent from my iPhone
Luca Milanesio
Skype: lucamilanesio

--

Olivier Dagenais

unread,
Nov 12, 2013, 9:04:40 AM11/12/13
to jenkin...@googlegroups.com
Hi,

I learned of this situation last night.  I'm the [current] maintainer of the tfs-plugin and I don't see commits since I released version 2.0, nor do I see a 'recovery' branch at https://github.com/jenkinsci/tfs-plugin

The machine I last made release 3.0.1 from has a clone with the 24 missing commits, and it looks like I should be able to just push them.

I don't want to interfere with any mass-fixing attempts discussed here (especially by Kohsuke), so please let me know if I should proceed.

Thanks!
- Oli
P.S.: I'm the one who released the last rake-plugin (which sort of makes me a maintainer), but the last commit to that repo was long enough ago that it wasn't affected.

Andrew Melo

unread,
Nov 12, 2013, 9:05:36 AM11/12/13
to jenkin...@googlegroups.com

On Tue, Nov 12, 2013 at 2:04 AM, Wes Turner <wes.t...@gmail.com> wrote:
There are fine-grained "File and branch conditions" in mercurial-server
that can prevent these sorts of pushes from succeeding. [1]

[1] http://dev.lshift.net/paul/mercurial-server/docbook.html#id1333360

The same functionality exists in git, just not in github.

-Andrew


--
--
Andrew Melo

Kohsuke Kawaguchi

unread,
Nov 12, 2013, 11:06:17 AM11/12/13
to jenkin...@googlegroups.com
GitHub has the event API that shows activities in a given repository and organization. The documentation page says only up to 300 can be retrieved, and so the events in the org no longer seems to contain the push from luca.

But when I look at the repository,  I can still see those pushes. For example, https://api.github.com/repos/jenkinsci/scoring-load-balancer-plugin/events

I think a little bit of scripting would allow us to go through each repository and find the trace of "git push -f" accurately.


2013/11/12 Luca Milanesio <luca.mi...@gmail.com>



--
Kohsuke Kawaguchi

Luca Milanesio

unread,
Nov 12, 2013, 2:35:36 PM11/12/13
to jenkin...@googlegroups.com
I've checked as well and tfs-plugin was in the list of recovered repos but I can't find the recovery branch either:

Will run a quick script for assess the situation with the others.

In the meantime I've extracted the lists of repos with pre-push and post-push SHA1 are the same (54 repos):
artifactory-plugin
branch-api-plugin
build-timeout-plugin
ci-game-plugin
clearcase-plugin
clearcase-ucm-plugin
collabnet-plugin
collapsing-console-sections-plugin
compress-artifacts-plugin
copy-project-link-plugin
copy-to-slave-plugin
credentials-plugin
crowd-plugin
crowd2-plugin
disable-failed-job-plugin
ec2-plugin
email-ext-plugin
gerrit-trigger-plugin
git-chooser-alternative-plugin
gitbucket-plugin
groovy-postbuild-plugin
humbug-plugin
jacoco-plugin
job-poll-action-plugin
jobConfigHistory-plugin
json-lib
leiningen-plugin
logfilesizechecker-plugin
mailer-plugin
maven-hpi-plugin
maven-info-plugin
nerrvana-plugin
next-build-number-plugin
next-executions-plugin
perforce-plugin
pitmutation-plugin
promoted-builds-plugin
prqa-plugin
rvm-plugin
scm-api-plugin
scm2job-plugin
scoring-load-balancer-plugin
selenium-tests
skype-im-plugin
ssh-agent-plugin
ssh-credentials-plugin
ssh-slaves-plugin
stashnotifier-plugin
subversion-plugin
swarm-plugin
transifex-plugin
vstestrunner-plugin
warnings-plugin
ws-cleanup-plugin

Will check now their master SHA1 and compare with the one "post-push" to see if the plugins maintainers have already pushed it or not.

Luca.

Kohsuke Kawaguchi

unread,
Nov 12, 2013, 2:44:15 PM11/12/13
to jenkin...@googlegroups.com

The tfs-plugin repository that I obtained yesterday does contain the
recovery branch pointing at f61f3fc4a395c40196fd51838bd4dc07b87bd139,
which is the tip of the master before luca's commit according to
https://api.github.com/repos/jenkinsci/tfs-plugin/events

git ls-remote that I just run also agrees with that:

> % git ls-remote https://github.com/jenkinsci/tfs-plugin.git
> 12b7198ce3b9ca2c940f3d995e64c481b22069b6 HEAD
> 12b7198ce3b9ca2c940f3d995e64c481b22069b6 refs/heads/master
> f61f3fc4a395c40196fd51838bd4dc07b87bd139 refs/heads/recovery
> d20b66b6f2af787faf1810f3e3af70bc354caa4c refs/heads/svn
> af49749cda406363888b3fd371193de0d1fe3d72 refs/heads/useTfsSdk
> 341fd55cc2765aee9a893d03f7a1fab507b86925 refs/meta/config
...

It's probably just not showing up in the UI presumably because the
recovery branch was created by an unusual means.
>> <mailto:jenkinsci-de...@googlegroups.com>.
>> For more options, visit https://groups.google.com/groups/opt_out.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkinsci-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


Luca Milanesio

unread,
Nov 12, 2013, 2:58:18 PM11/12/13
to jenkin...@googlegroups.com, jenkin...@googlegroups.com
Ah ok :-) that is clarified then !

I'm looking now the list of 54 repos to see if they have been already fixed by their maintainers ....

Will provide further updates later.

Luca
---------
Sent from my iPhone
Luca Milanesio
Skype: lucamilanesio


Slide

unread,
Nov 12, 2013, 3:01:38 PM11/12/13
to Jenkins Dev
Please don't run any more scripts :-). The email-ext-plugin is just fine, please take it off your list for needing recovery.

Luca Milanesio

unread,
Nov 12, 2013, 3:27:33 PM11/12/13
to jenkin...@googlegroups.com, Jenkins Dev
I'm just checking using the GitHub  GUI, will not push anything :-) 

I am trying the get the final list of repos that would need further fixing. Then would leave to the plugin maintainer to judge which is the latest, as it was not detected by the GitHub support team.

Luca
---------
Sent from my iPhone
Luca Milanesio
Skype: lucamilanesio

Hugo Arès

unread,
Nov 12, 2013, 3:37:53 PM11/12/13
to jenkin...@googlegroups.com
The git-plugin is missing at least one commit:

commit 1e894cd9ce564c2ed8e1283b091b1105512efcdd
Author: Stephen Connolly <...>
Date:   Fri Nov 8 15:51:49 2013 +0000

    switch to c:select to support in-place addition of credentials

Hugo Arès

unread,
Nov 12, 2013, 3:45:59 PM11/12/13
to jenkin...@googlegroups.com
The build-pipeline-plugin is missing the last 2 commits:

commit 220610b7683b7ca40b6b470bc0b33191c140841a
Merge: 64db20c d9a6545
Author: geoffbullen <...>
Date:   Mon Nov 4 14:25:50 2013 -0800

    Merge pull request #18 from jglick/lower-core-dep
   
    Drop core dependency to 1.509 (LTS line).

commit d9a6545b7cac1de115f395cf234bbf1637ce22e2
Author: Jesse Glick <...>
Date:   Mon Nov 4 17:18:01 2013 -0500

    Drop core dependency to 1.509 (LTS line).
    Current code requires at least 1.489 (for 15ba345b), but 1.521 seems unjustified.

Kohsuke Kawaguchi

unread,
Nov 12, 2013, 3:51:27 PM11/12/13
to jenkin...@googlegroups.com

Hi, Hugo,

From what I can tell, those repositories are in the CSV file provided
by GitHub, and their commits do seem to be in the 'recovery' branch. So
those are already accounted for.

Right now we are focusing on either repositories that are not in the CSV
file, or the commits that are not in the 'recovery' branch.


On 11/12/2013 12:45 PM, Hugo Ar�s wrote:
> The build-pipeline-plugin is missing the last 2 commits:
>
> commit 220610b7683b7ca40b6b470bc0b33191c140841a
> Merge: 64db20c d9a6545
> Author: geoffbullen <...>
> Date: Mon Nov 4 14:25:50 2013 -0800
>
> Merge pull request #18 from jglick/lower-core-dep
>
> Drop core dependency to 1.509 (LTS line).
>
> commit d9a6545b7cac1de115f395cf234bbf1637ce22e2
> Author: Jesse Glick <...>
> Date: Mon Nov 4 17:18:01 2013 -0500
>
> Drop core dependency to 1.509 (LTS line).
> Current code requires at least 1.489 (for 15ba345b), but 1.521
> seems unjustified.
>
> On Tuesday, November 12, 2013 3:37:53 PM UTC-5, Hugo Ar�s wrote:
>
> The git-plugin is missing at least one commit:
>
> commit 1e894cd9ce564c2ed8e1283b091b1105512efcdd
> Author: Stephen Connolly <...>
> Date: Fri Nov 8 15:51:49 2013 +0000
>
> switch to c:select to support in-place addition of credentials
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkinsci-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


--

Kohsuke Kawaguchi

unread,
Nov 12, 2013, 3:55:54 PM11/12/13
to jenkin...@googlegroups.com

Indeed, looking at
https://api.github.com/repos/jenkinsci/scoring-load-balancer-plugin/events,
I see that Luca's push had changed master from
0ca553b933521ba82746a491b3c29d8c6cc2ae89 to
634609e0e7ffe754f1ec3fed8871a89f7403508b.

So there's something wrong with the list provided by GitHub...
> --
> You received this message because you are subscribed to the Google
> Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jenkinsci-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.


Kohsuke Kawaguchi

unread,
Nov 12, 2013, 4:22:08 PM11/12/13
to jenkin...@googlegroups.com

Since this CSV file and the 'recovery' branch created by GitHub support
doesn't appear to be complete, I'm putting this on hold for now.
Reply all
Reply to author
Forward
0 new messages