gclient runhooks fails on Android.

2,798 views
Skip to first unread message

soh...@chromium.org

unread,
Oct 31, 2017, 5:23:12 AM10/31/17
to Chromium-dev
Hi,

gclient runhooks seems to be failing on ToT for Chrome on Android. This was working till sometime back (2 weeks).

Any pointers what needs to done to fix this ? My Linux sync is going fine.


Syncing projects: 100% (106/106), done.                                      
Running hooks:  23% (16/68) gn_linux64       
________ running '/usr/bin/python src/third_party/depot_tools/download_from_google_storage.py --no_resume --no_auth --bucket chromium-gn -s src/buildtools/linux64/gn.sha1' in '/home/sohan/Chrome_Android'
Failed to fetch file gs://chromium-gn/47db65960178d60c859ca1d21de544ca4a2c01ae for src/buildtools/linux64/gn. [Err: /home/sohan/depot_tools/vpython: line 12: /home/sohan/depot_tools/.cipd_bin/vpython: No such file or directory
]
0> Failed to fetch file gs://chromium-gn/47db65960178d60c859ca1d21de544ca4a2c01ae for src/buildtools/linux64/gn, skipping. [Err: /home/sohan/depot_tools/vpython: line 12: /home/sohan/depot_tools/.cipd_bin/vpython: No such file or directory
]
Downloading 1 files took 0.368585 second(s)
Error: Command '/usr/bin/python src/third_party/depot_tools/download_from_google_storage.py --no_resume --no_auth --bucket chromium-gn -s src/buildtools/linux64/gn.sha1' returned non-zero exit status 1 in /home/sohan/Chrome_Android


Thanks,
Sohan

Daniel Jacques

unread,
Oct 31, 2017, 8:36:00 AM10/31/17
to Chromium-dev, Robbie Iannucci
It looks like your CIPD bootstrap has broken. Not sure what would have caused that, since the process is supposed to be self-repairing.

The simple fix should be:
$ git -C /path/to/depot_tools clean -f -x -d
$ /path/to/depot_tools/cipd

However, before running that, can you run and post the output of:
$ /path/to/depot_tools/update_depot_tools

... for analysis? Something should be going wrong there.

soh...@chromium.org

unread,
Oct 31, 2017, 8:52:47 AM10/31/17
to Chromium-dev, iann...@chromium.org
Hey,

Thanks for the info. But the same runhooks error still persists.
Please see the o/p inline.

Thanks,
Sohan

On Tuesday, October 31, 2017 at 2:36:00 PM UTC+2, Daniel Jacques wrote:
It looks like your CIPD bootstrap has broken. Not sure what would have caused that, since the process is supposed to be self-repairing.

The simple fix should be:
$ git -C /path/to/depot_tools clean -f -x -d
$ /path/to/depot_tools/cipd

The clean cmd does remove everything indeed.
After running depot_tools/cipd i get the following message,

However, before running that, can you run and post the output of:
$ /path/to/depot_tools/update_depot_tools

... for analysis? Something should be going wrong there.

 update_depot_tools returns after printing

Daniel Jacques

unread,
Oct 31, 2017, 9:00:24 AM10/31/17
to Chromium-dev, iann...@chromium.org
Does this directory exist?  /home/sohan/depot_tools/.cipd_bin/
Are there any files in it?

The script that should be making everything work is cipd_bin_setup.sh. You can run it directly via:
$ bash
$ source /path/to/depot_tools/cipd_bin_setup.sh
$ cipd_bin_setup

Does that produce any output? Does your ".cipd_bin" directory contain the "vpython" binary?

If that doesn't work, you can try deconstructing that script and running the command manually:
$ /path/to/depot_tools/cipd ensure -log-level debug -ensure-file /path/to/depot_tools/cipd_manifest.txt -root /path/to/depot_tools/.cipd_bin

Does that produce any output? Does your ".cipd_bin" directory contain the "vpython" binary?

The end-game of that sequence of scripts is to initialize the contents of the /home/sohan/depot_tools/.cipd_bin/ folder, which should include the "vpython" binary that your original error message listed as missing. If those aren't being initialized, some debug output should be visible.

soh...@chromium.org

unread,
Oct 31, 2017, 9:35:50 AM10/31/17
to Chromium-dev, iann...@chromium.org
Hi,

Thanks for checking.
But unfortunately, neither the ".cipd_bin" directory is created, nor the vpyhton inside, once i source and run cipd_bin_setup.sh. But i can see a vpython in depot_tools/ folder.

There isn't really much o/p except for that 1line, even if i deconstruct and run the script.
Any environment modification is reqd to run the cipd_bin_setup ?

Thanks,
Sohan

Daniel Jacques

unread,
Oct 31, 2017, 10:09:13 AM10/31/17
to Chromium-dev, iann...@chromium.org
So there are (for good and bad reasons) two different levels of bootstrapping here: one to obtain the "cipd" binary (//depot_tools/cipd), depositing it at //depot_tools/.cipd_client, and another to use that binary to bootstrap additional binaries (cipd_bin_setup.sh), depositing at //depot_tools/.cipd_bin. If you simply look at it, //depot_tools/vpython is a bootstrap script that reaches into //depot_tools/.cipd_bin and runs "vpython" from there; since the problem seems to be happening before the binary exists, that script is not relevant.

The only scenario that I can think of that would account for your output (or lack thereof) is for the first stage bootstrap to be failing.

Can you:
  • Check if the bootstrapped client binary exists at: /path/to/depot_tools/.cipd_client
  • If so, can you get its version: /path/to/depot_tools/.cipd_client version
If not, the problem is likely with the //depot_tools/cipd bootstrap script. Can you edit it, adding "set -x" to the top, right above "set -e -o pipefail". It should look something like:

#!/bin/bash -e

# Copyright...

set -x
set -e -o pipefail

MYPATH=...

Then, run /path/to/depot_tools/cipd and paste the output. Something seems to be going wrong in that area.

Thanks!

soh...@chromium.org

unread,
Oct 31, 2017, 10:41:50 AM10/31/17
to Chromium-dev, iann...@chromium.org
Hi,

Thanks again.
Looks like i do have a cipd_client_version file, its version is,

git_revision:d2677a4477e59cb7de00f1fb8a00e96b1aaeb927

But no depot_tools/.cipd_client file.

And following is the output while running cipd cmd after adding "set -x"

└─ $ ▶ cipd
+ set -e -o pipefail
++ dirname /home/sohan/depot_tools/cipd
+ MYPATH=/home/sohan/depot_tools
++ cat /home/sohan/depot_tools/cipd_client_version
+ : git_revision:d2677a4477e59cb7de00f1fb8a00e96b1aaeb927
++ uname -s
++ tr '[:upper:]' '[:lower:]'
+ UNAME=linux
+ case $UNAME in
+ PLAT=linux
++ uname -m
++ tr '[:upper:]' '[:lower:]'
+ UNAME=x86_64
+ case $UNAME in
+ ARCH=amd64
+ CLIENT=/home/sohan/depot_tools/.cipd_client
++ git -C /home/sohan/depot_tools rev-parse HEAD
+ USER_AGENT=depot_tools/7d1c484ecd93e94c5258dc330602ac37c5efab44
+ '[' '!' -e /home/sohan/depot_tools/.cipd_client ']'
++ mktemp -p /home/sohan/depot_tools
+ CIPD_CLIENT_TMP=/home/sohan/depot_tools/tmp.qOM88G9qQL
+ hash curl
+ curl 'https://chrome-infra-packages.appspot.com/client?platform=linux-amd64&version=git_revision:d2677a4477e59cb7de00f1fb8a00e96b1aaeb927' -s --show-error -f -A depot_tools/7d1c484ecd93e94c5258dc330602ac37c5efab44 -L -o /home/sohan/depot_tools/tmp.qOM88G9qQL

Thanks,
Sohan

Daniel Jacques

unread,
Oct 31, 2017, 12:06:22 PM10/31/17
to Chromium-dev, iann...@chromium.org
I was actually asking you to run "/path/to/depot_tools/.cipd_client version", not check for the "cipd_client_version" file, but that's OK - the missing ".cipd_client" file is a sufficient sign that the bootstrap is not working.

From the looks of things, your script is stopping after the "curl" command, which indicates that it is returning a non-zero error status. Can you try running a modified version of it in verbose mode:


DO NOT paste the output directly, as it may contain sensitive information; however, examine it and see if you're getting any errors and, if so, include the error text, any relevant error context, and the exit code as long as it isn't sensitive.

FWIW that command works for me and returns a zero exit code.

Sohan Jyoti Ghosh

unread,
Oct 31, 2017, 12:42:25 PM10/31/17
to Chromium-dev, iann...@chromium.org
Indeed there is some curl certificate problem, but since gclient sync was going through fine, i was hoping runhooks should pass too.
I will try to fix the curl problem and check again.

Just a note if i run curl with -k option in your command it does fetch a lot of data (not pasting here).
I will fix the curl SSL issue and reply.

Thanks for checking.
Sohan

Daniel Jacques

unread,
Oct 31, 2017, 1:22:24 PM10/31/17
to Chromium-dev, iann...@chromium.org
Good to hear you found the problem.

It's a shame that everything had the appearance of succeeding when, in reality, some substeps were failing. Feel free to file an Infra bug, referencing this thread, to look into how to make user-side script expectation violations more visible to the user.

Shubham Bansal

unread,
Nov 11, 2017, 7:59:04 AM11/11/17
to d...@google.com, Chromium-dev, iann...@chromium.org
Hi,

I am also getting the same error.

Traceback (most recent call last):
File "/home/in3o/chromium/depot_tools/gclient.py", line 2682, in <module>
sys.exit(main(sys.argv[1:]))
File "/home/in3o/chromium/depot_tools/gclient.py", line 2668, in main
return dispatcher.execute(OptionParser(), argv)
File "/home/in3o/chromium/depot_tools/subcommand.py", line 252, in execute
return command(parser, args[1:])
File "/home/in3o/chromium/depot_tools/gclient.py", line 2517, in CMDrunhooks
return client.RunOnDeps('runhooks', args)
File "/home/in3o/chromium/depot_tools/gclient.py", line 1513, in RunOnDeps
self.RunHooksRecursively(self._options, pm)
File "/home/in3o/chromium/depot_tools/gclient.py", line 1033, in
RunHooksRecursively
hook.run(self.root.root_dir)
File "/home/in3o/chromium/depot_tools/gclient.py", line 219, in run
cmd, cwd=cwd, always=self._verbose)
File "/home/in3o/chromium/depot_tools/gclient_utils.py", line 314,
in CheckCallAndFilterAndHeader
return CheckCallAndFilter(args, **kwargs)
File "/home/in3o/chromium/depot_tools/gclient_utils.py", line 509,
in CheckCallAndFilter
**kwargs)
File "/home/in3o/chromium/depot_tools/subprocess2.py", line 262, in __init__
% (str(e), kwargs.get('cwd'), args[0]))
OSError: Execution failed with error: [Errno 2] No such file or directory.
Check that /home/in3o/chromium or vpython exist and have execution permission.


=========================================
here is the output for .cipd_client version.
06:25 PM iN3O:chromium>./depot_tools/.cipd_client version
Package name: infra/tools/cipd/linux-amd64
Instance ID: 4335bff1ec7259c5081b636ab312dcf6b9c6dd4b

Can anybody help me with this?
Best,
Shubham Bansal
> --
> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev
> ---
> You received this message because you are subscribed to the Google Groups
> "Chromium-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chromium-dev...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/e0c1e84b-eb42-49c1-ae73-d40071835a07%40chromium.org.

Shubham Bansal

unread,
Nov 11, 2017, 8:17:13 AM11/11/17
to d...@google.com, Chromium-dev, iann...@chromium.org
Okay. Found the issue.

I added relative depot_tools path to the PATH.

Sorry, my mistake.
Best,
Shubham Bansal

林振章

unread,
Nov 11, 2017, 9:01:49 PM11/11/17
to illusio...@gmail.com, d...@google.com, Chromium-dev, iann...@chromium.org
>> email to chromium-dev+unsubscribe@chromium.org.

>> To view this discussion on the web visit
>> https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/e0c1e84b-eb42-49c1-ae73-d40071835a07%40chromium.org.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAHgaXdL39aKZ-ht6kaYdSruLdabhkUz%3D0ULiiWF1JiquptnZ_g%40mail.gmail.com.

Message has been deleted

Robert Iannucci

unread,
Mar 24, 2018, 2:40:58 PM3/24/18
to 龚泽龙, Chromium-dev, d...@google.com
Replied in the other thread, but if you're getting connection refusal from https://chrome-infra-packages.appspot.com, is it possible that your network is blocking it somehow?

On Sat, Mar 24, 2018 at 7:07 AM 龚泽龙 <gongz...@gmail.com> wrote:
Hi, Shubham Bansal 

How do you fix the issue? Just added depot_tools path to the PATH or anything else?

I met the same issue, and I cannot fix it by myself.

gongzelong@gongzelong-ThinkPad-X260:~/Documents/git/ofa/mobile/mobile$ cipd ensure -log-level debug -ensure-file path/to/depot_tools/cipd_manifest.txt -root path/to/depot_tools/.cipd_bin
Bootstrapping cipd client for linux-amd64 from https://chrome-infra-packages.appspot.com/client?platform=linux-amd64&version=git_revision:ae28364c740acff97ae118adcb2808b6cb5129c5...
curl: (7) Couldn't connect to server

龚泽龙

unread,
Mar 29, 2018, 4:03:59 AM3/29/18
to Robert Iannucci, Chromium-dev, d...@google.com
I can see the url chrome-infra-packages.appspot.com in browser, and ping is okay.

I do not use shadowsocks as proxy. I use PulseVPN, but I do not know what is the root cause.

gongzelong@gongzelong-ThinkPad-X260:~/Documents/git/depot_tools$ ./update_depot_tools 
Bootstrapping cipd client for linux-amd64 from https://chrome-infra-packages.appspot.com/client?platform=linux-amd64&version=git_revision:4d19637ec2c3d1efd8c6a1b05285118b786919e2...

curl: (7) Couldn't connect to server



curl -v 'https://chrome-infra-packages.appspot.com/client?platform=linux-amd64&version=git_revision:4d19637ec2c3d1efd8c6a1b05285118b786919e2' -f -L -o /dev/null; echo $?


gongzelong@gongzelong-ThinkPad-X260:~/Documents/git/depot_tools$ curl -v 'https://chrome-infra-packages.appspot.com/client?platform=linux-amd64&version=git_revision:4d19637ec2c3d1efd8c6a1b05285118b786919e2' -f -L -o /dev/null; echo $?
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 172.217.160.84...
* Trying 2404:6800:4008:803::2014...
* Immediate connect fail for 2404:6800:4008:803::2014: Network is unreachable
* Connected to chrome-infra-packages.appspot.com (172.217.160.84) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 604 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: *.appspot.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: C=US,ST=California,L=Mountain View,O=Google Inc,CN=*.appspot.com
* start date: Tue, 13 Mar 2018 19:19:59 GMT
* expire date: Tue, 05 Jun 2018 18:18:00 GMT
* issuer: C=US,O=Google Trust Services,CN=Google Internet Authority G3
* compression: NULL
* ALPN, server accepted to use http/1.1
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0> GET /client?platform=linux-amd64&version=git_revision:4d19637ec2c3d1efd8c6a1b05285118b786919e2 HTTP/1.1
> Host: chrome-infra-packages.appspot.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< Content-Type: text/html; charset=utf-8
< Cache-Control: no-cache
< Content-Security-Policy: child-src https://accounts.google.com; default-src 'self'; font-src 'self' https://fonts.gstatic.com; img-src 'self' https://www.google-analytics.com https://*.googleusercontent.com; object-src 'none'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://www.google-analytics.com https://www.google.com/jsapi https://apis.google.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< X-Frame-Options: DENY
< Location: https://storage.googleapis.com/chrome-infra-packages/store/SHA1/cd9ffccbff38cb3dc5699450e6cf49e10c965895?GoogleAccessId=chrome-infra-packages%40appspot.gserviceaccount.com&Expires=1522316597&Signature=CpZWF1xYVDlqRckji144R3fkZZQTN6pwjA5IFGsY2QORAIaebMTcyqwe1l91KVkenCOTj7ok4xp7RyrdJ6sp%2F%2FwvBNTWnSEU4z8zRKDUo%2BoIT%2B03XDvSJ2841h9U4ABmPL1gFrFLiqEzm1hfMKGOBSq1eerGpIZFERVW72h70X0n2bBE1jtXpW1Y1wj%2F0rwVAXXy1S21Lw7pRb0tQAItop1QHqT2LuuVrpY8PuUilA%2BO9D7Eg12OgOVH7tS8NpzDnG%2B4VGWcFKZ5n5xn7qJd8qpDGbU1qpKlbR8ZksQxUQa1C6Mlw%2BIbIRgxXTayQ5Lix9y1g8iIrqbeGESNXkXjjw%3D%3D&response-content-disposition=attachment%3B+filename%3D%22cipd%22
< X-Cloud-Trace-Context: 6898b32557befc4276ddd00494cdb6d3
< Date: Thu, 29 Mar 2018 07:53:13 GMT
< Server: Google Frontend
< Content-Length: 0
< Alt-Svc: hq=":443"; ma=2592000; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=":443"; ma=2592000; v="42,41,39,35"
< 
0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0
* Connection #0 to host chrome-infra-packages.appspot.com left intact
* Issue another request to this URL: 'https://storage.googleapis.com/chrome-infra-packages/store/SHA1/cd9ffccbff38cb3dc5699450e6cf49e10c965895?GoogleAccessId=chrome-infra-packages%40appspot.gserviceaccount.com&Expires=1522316597&Signature=CpZWF1xYVDlqRckji144R3fkZZQTN6pwjA5IFGsY2QORAIaebMTcyqwe1l91KVkenCOTj7ok4xp7RyrdJ6sp%2F%2FwvBNTWnSEU4z8zRKDUo%2BoIT%2B03XDvSJ2841h9U4ABmPL1gFrFLiqEzm1hfMKGOBSq1eerGpIZFERVW72h70X0n2bBE1jtXpW1Y1wj%2F0rwVAXXy1S21Lw7pRb0tQAItop1QHqT2LuuVrpY8PuUilA%2BO9D7Eg12OgOVH7tS8NpzDnG%2B4VGWcFKZ5n5xn7qJd8qpDGbU1qpKlbR8ZksQxUQa1C6Mlw%2BIbIRgxXTayQ5Lix9y1g8iIrqbeGESNXkXjjw%3D%3D&response-content-disposition=attachment%3B+filename%3D%22cipd%22'
0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0* Trying 2404:6800:4012::2010...
* Immediate connect fail for 2404:6800:4012::2010: Network is unreachable
* Closing connection 1

curl: (7) Couldn't connect to server
7


gongzelong@gongzelong-ThinkPad-X260:~/Documents/git/depot_tools$ ping chrome-infra-packages.appspot.com
PING appspot-preview.l.google.com (216.58.200.244) 56(84) bytes of data.
64 bytes from tsa03s01-in-f244.1e100.net (216.58.200.244): icmp_seq=1 ttl=47 time=414 ms
64 bytes from tsa03s01-in-f244.1e100.net (216.58.200.244): icmp_seq=2 ttl=47 time=391 ms
64 bytes from tsa03s01-in-f244.1e100.net (216.58.200.244): icmp_seq=3 ttl=47 time=400 ms
64 bytes from tsa03s01-in-f244.1e100.net (216.58.200.244): icmp_seq=4 ttl=47 time=455 ms
64 bytes from tsa03s01-in-f244.1e100.net (216.58.200.244): icmp_seq=5 ttl=47 time=391 ms
64 bytes from tsa03s01-in-f244.1e100.net (216.58.200.244): icmp_seq=6 ttl=47 time=386 ms
^C
--- appspot-preview.l.google.com ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5033ms
rtt min/avg/max/mdev = 386.829/406.983/455.585/23.552 ms

Yang Liu

unread,
Mar 3, 2020, 4:19:14 PM3/3/20
to Chromium-dev, iann...@chromium.org, d...@google.com
Hi, I meet the same problem now. Have your solved this issue?

在 2018年3月29日星期四 UTC+8下午4:03:59，龚泽龙写道：
Reply all
Reply to author
Forward
0 new messages