Global DDoS on DynDNS and noise created today on Gerrit Verified labels

[lucamilanesio]

Hi Gerrit Contributors and Maintainers,

here has been today a bit on "noise" of GerritForge Verified -1 on Gerrit changes, this has been caused by three problems:

1) The global DDoS on DynDNS has created random failures of resolution to GitHub ... and you know that nowadays when GitHub is down or not reachable there is always a build in the world that would fails for that

2) gerrit-review.googlesource.com stopped for a few hours to provide the Verified status on the changes list REST-API. I thought it was a bug on my Groovy script and, stupidly, just ignored the NULL and thought it was worth doing another build: this created a "build loop" on changes

3) I've enabled the compilation with Buck + Java7 vs. Buck + Java8, using a specific set-buck.sh script. Obviously this wasn't called by the verifier job which failed multiple times with "buck command not found"

Bottom line: a perfect storm just created a lot of dust cloud.

Now situation should have come back to normal: I am just clearing up the invalid "Verified -1" that your change may have received in the past few hours.

Apologies for the noise, at least I can say that 1) wasn't my or our fault :-)

Luca.

[lucamilanesio]

Actually, 2) is still outstanding: see below.

$ curl 'https://gerrit-review.googlesource.com/changes/?n=1&O=81'

)]}'

[

  {

    "id": "gerrit~master~Id45967aa844faedee3ab501491cd30cdb903a789",

    "project": "gerrit",

    "branch": "master",

    "hashtags": [],

    "change_id": "Id45967aa844faedee3ab501491cd30cdb903a789",

    "subject": "bazel: build a polygerrit war too.",

    "status": "NEW",

    "created": "2016-10-20 12:58:07.000000000",

    "updated": "2016-10-21 21:49:34.000000000",

    "submit_type": "MERGE_IF_NECESSARY",

    "mergeable": true,

    "insertions": 20,

    "deletions": 5,

    "_number": 89327,

    "owner": {

      "_account_id": 1024147,

      "name": "Han-Wen Nienhuys",

      "email": "han...@google.com",

      "avatars": [

        {

          "url": "https://lh6.googleusercontent.com/-6s-DM08KjUU/AAAAAAAAAAI/AAAAAAAAAAA/9TTAkk4Pup0/s26-p/photo.jpg",

          "height": 26

        },

        {

          "url": "https://lh6.googleusercontent.com/-6s-DM08KjUU/AAAAAAAAAAI/AAAAAAAAAAA/9TTAkk4Pup0/s32-p/photo.jpg",

          "height": 32

        },

        {

          "url": "https://lh6.googleusercontent.com/-6s-DM08KjUU/AAAAAAAAAAI/AAAAAAAAAAA/9TTAkk4Pup0/s100-p/photo.jpg",

          "height": 100

        }

      ]

    },

    "labels": {},

    "_more_changes": true

  }

]

Is that normal that labels is an empty object?

[David Pursehouse]

Missing labels was due to a regression that has since been fixed.  See: https://bugs.chromium.org/p/gerrit/issues/detail?id=4802

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luca Milanesio]

Ah, cool: this explains why sometimes they appear and sometimes are invisible.

I guess Googlers are doing the rolling upgrade of the nodes ...

Luca.

[lucamilanesio]

For 1) it seems that our Polymer build relies on the fact of GitHub being reachable:

00:05:36.497 OSError: Command failed: ['/home/jenkins/workspace/Gerrit-stable-2.13/buck-out/gen/tools/js/run_npm_binary.pex', '/home/jenkins/workspace/Gerrit-stable-2.13/buck-out/gen/lib/js/bower/bo...@1.7.9.npm_binary.tgz', '-l=error', '-j', 'info', 'polymer/polymer#1.4.0']
00:05:36.524 [{
00:05:36.566   "code": "ECMDERR",
00:05:36.605   "details": "fatal: unable to connect to github.com:\ngithub.com: Temporary failure in name resolution\n\n",
00:05:36.625   "exitCode": 128,
00:05:36.664   "data": {
00:05:36.687     "endpoint": {
00:05:36.705       "name": "",
00:05:36.717       "source": "polymer/polymer",
00:05:36.732       "target": "1.4.0"
00:05:36.751     },
00:05:36.772     "resolver": {
00:05:36.800       "name": "polymer",
00:05:36.847       "source": "git://github.com/polymer/polymer.git",
00:05:36.872       "target": "1.4.0"
00:05:36.886     }
00:05:36.898   },
00:05:36.912   "id": "ECMDERR",
00:05:36.952   "level": "error",
00:05:36.966   "message": "Failed to execute \"git ls-remote --tags --heads git://github.com/polymer/polymer.git\", exit code of #128\nfatal: unable to connect to github.com:\ngithub.com: Temporary failure in name resolution\n\n",

... unfortunately this does not happen often, as GitHub outages are quite frequent :-(

Is there a way to mirror it to Gerrit-Review?

Luca.

On Friday, October 21, 2016 at 10:58:40 PM UTC+1, lucamilanesio wrote:

[Luca Milanesio]

I still don't see the review status on the changes list:

Has the fix being rolled out to Gerrit-Review?

Luca.

On 22 Oct 2016, at 00:05, David Pursehouse <david.pu...@gmail.com> wrote: