fetch webrtc fails...

[Ed Sutter]

Hi, 

Just getting started here.

My host system is ubuntu 14.04 LTS.

Trying to follow the instructions on http://www.webrtc.org/native-code/development#TOC-Before-you-start.

I've installed the prerequisite software & depot_tools, next step "fetch webrtc" runs for about an hour

then dies (see below). 

Are there additional prequequisites?

Tx in advance,

Ed

...

Trying to download prebuilt clang

clang 223108 unpacked

Hook '/usr/bin/python src/tools/clang/scripts/update.py --if-needed' took 25.80 secs

________ running '/usr/bin/python src/build/util/lastchange.py -o src/build/util/LASTCHANGE' in '/home/els/webrtc/webrtc/src/chromium'

________ running '/usr/bin/python src/build/util/lastchange.py -s src/third_party/WebKit -o src/build/util/LASTCHANGE.blink' in '/home/els/webrtc/webrtc/src/chromium'

________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-gn -s src/buildtools/win/gn.exe.sha1' in '/home/els/webrtc/webrtc/src/chromi

um'

________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-gn -s src/buildtools/mac/gn.sha1' in '/home/els/webrtc/webrtc/src/chromium'

________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s src/buildtools/linux32/gn.sha1' in '/home/els/webrtc/webrtc/src/chrom

ium'

Error: Command download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s src/buildtools/linux32/gn.sha1 returned non-zero exit status 1 in /home/

els/webrtc/webrtc/src/chromium

File gs://chromium-gn/2e23cfba3baf503b9f81d3365c4f09770ff48a9a for src/buildtools/linux32/gn does not exist.

0> File gs://chromium-gn/2e23cfba3baf503b9f81d3365c4f09770ff48a9a for src/buildtools/linux32/gn does not exist, skipping.

Hook 'download_from_google_storage --no_resume '--platform=linux*' --no_auth --bucket chromium-gn -s src/buildtools/linux32/gn.sha1' took 421.42 secs

Error: Command /usr/bin/python -u src/sync_chromium.py --target-revision 271c6cca48a6cef32c0f3add3b17b700707deec5 returned non-zero exit status 2 in /home/els/webrtc/webrtc

Hook '/usr/bin/python -u src/sync_chromium.py --target-revision 271c6cca48a6cef32c0f3add3b17b700707deec5' took 4317.85 secs

Traceback (most recent call last):

  File "/home/els/bin/depot_tools/fetch.py", line 334, in <module>

    sys.exit(main())

  File "/home/els/bin/depot_tools/fetch.py", line 330, in main

    return run(options, spec, root)

  File "/home/els/bin/depot_tools/fetch.py", line 324, in run

    return checkout.init()

  File "/home/els/bin/depot_tools/fetch.py", line 169, in init

    super(GclientGitSvnCheckout, self).init()

  File "/home/els/bin/depot_tools/fetch.py", line 136, in init

    self.run_gclient(*sync_cmd)

  File "/home/els/bin/depot_tools/fetch.py", line 76, in run_gclient

    return self.run(cmd_prefix + cmd, **kwargs)

  File "/home/els/bin/depot_tools/fetch.py", line 66, in run

    return subprocess.check_call(cmd, **kwargs)

  File "/usr/lib/python2.7/subprocess.py", line 540, in check_call

    raise CalledProcessError(retcode, cmd)

subprocess.CalledProcessError: Command '('gclient', 'sync')' returned non-zero exit status 2

[Ed Sutter]

Ok, I'm replying to my own question here...

Can't explain why I needed to do this; however it appears that the problem is related to the fact that I am behind a proxy

and the --no_auth option is used (in depot_tools) when the download_from_google_storage.py script is called.  HUH??

Based on this https://github.com/GoogleCloudPlatform/gsutil/issues/241 I modified my copy of "download_from_google_storage.py"

so that the --no_auth option would have no affect.

I also created a ~/.boto file with three lines:

[Boto]

proxy = my.proxy.goes.here.com

proxy_port = PROXY_PORT_NUMBER

Then I re-ran "fetch webrtc" and it completed in about 75 minutes.

Go figure...

[ashish mehta]

Hi Ed,

As per your post i created a boto file and made changes in download_from_google_storage.py.

This has been the change i made in file(in red):
if options.no_auth:
    if (set(('http_proxy', 'https_proxy')).intersection(
        env.lower() for env in os.environ) and
        'NO_AUTH_BOTO_CONFIG' not in os.environ):
      print >> sys.stderr, ('NOTICE: You have PROXY values set in your '
                            'environment, but gsutil in depot_tools does not '
                            '(yet) obey them.')
      print >> sys.stderr, ('Also, --no_auth prevents the normal BOTO_CONFIG '
                            'environment variable from being used.')
      print >> sys.stderr, ('To use a proxy in this situation, please supply '
                            'those settings in a .boto file pointed to by '
                            'the NO_AUTH_BOTO_CONFIG environment var.')
    #options.boto = os.environ.get('NO_AUTH_BOTO_CONFIG', os.devnull)

Also find the ~/.boto file i created:
[Boto]
proxy = 10.135.80.164
proxy_port = 8678

And i also set:
export NO_AUTH_BOTO_CONFIG=~/.boto

But i am getting below error
Error: Command download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s src/buildtools/linux32/gn.sha1 returned non-zero exit status 1 in /home/ubuntu/src/chromium

File gs://chromium-gn/2e23cfba3baf503b9f81d3365c4f09770ff48a9a for src/buildtools/linux32/gn does not exist.
0> File gs://chromium-gn/2e23cfba3baf503b9f81d3365c4f09770ff48a9a for src/buildtools/linux32/gn does not exist, skipping.

Hook 'download_from_google_storage --no_resume '--platform=linux*' --no_auth --bucket chromium-gn -s src/buildtools/linux32/gn.sha1' took 293.65 secs
Error: Command /usr/bin/python -u src/sync_chromium.py --target-revision 9070a8059b513108b09d30f96576b5ce11d0857a returned non-zero exit status 2 in /home/ubuntu
Hook '/usr/bin/python -u src/sync_chromium.py --target-revision 9070a8059b513108b09d30f96576b5ce11d0857a' took 351.04 secs

Could you please tell me what i may be doing wrong.

Thanks
Ashish Mehta

[ashish mehta]

Hi Ed,

Can you please tell what where the changes you made in "download_from_google_storage.py" file.

Thanks
Ashish Mehta
On Wednesday, January 14, 2015 at 2:32:46 AM UTC+5:30, Ed Sutter wrote:

[Ed Sutter]

Hi,
That snippet isn't in my version of that script, so you must have something newer.
Seems like this stuff changes frequently.
Unfortunately I was just lucky (took a week of digging to find that luck!).  I don't
know much about what these scripts are doing; and I have not tried to use fetch
since I got it to work earlier.
Your experience it enforcing the idea that I should avoid that!
Sorry I can't help more, but please do me a favor and let me know if you figure it out,
I'll probably need it as well.

Regards,
Ed 

[Ed Sutter]

Hi,
The change I made was to force that script to ignore the incoming --no_auth option...
In my version of the script this was at line #386...
#  if options.no_auth:
#    options.boto = os.devnull

HTH
Ed 

[aab112...@hotmail.com]

ran into same problem when building in a vagrant vbox:

- depot_tools @ 455dc92f589da5173289bf20d4eb0ab78a790498
- webrtc @ 026b892e724c3f47bde92d773d84099768e57ec8
- Linux vagrant-ubuntu-trusty-64 3.13.0-44-generic #73-Ubuntu SMP Tue Dec 16 00:22:43 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

was fixed by applying this patch (thanks Ed) to disable download_from_google_storage.py --no-auth.

[lif...@gmail.com]

Hi, Ed and Ashish,

After I tried Ed's method, I still ran into the same problem while I'm trying to "gclient sync --force" WebRTC in Ubantu server system:

--------------------------------------- the Error inforamtion -----------------------------------

________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s src/buildtools/linux32/gn.sha1' in '/home/ubt-gl/wrtc-c/src/chromium'

Error: Command download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s src/buildtools/linux32/gn.sha1 returned non-zero exit status 1 in /home/ubt-gl/wrtc-c/src/chromium

File gs://chromium-gn/2e23cfba3baf503b9f81d3365c4f09770ff48a9a for src/buildtools/linux32/gn does not exist.

0> File gs://chromium-gn/2e23cfba3baf503b9f81d3365c4f09770ff48a9a for src/buildtools/linux32/gn does not exist, skipping.

Error: Command /usr/bin/python -u src/sync_chromium.py --target-revision 601e6f32704fed600914fd5198c24395c4dc36f5 returned non-zero exit status 2 in /home/ubt-gl/wrtc-c

Hook '/usr/bin/python -u src/sync_chromium.py --target-revision 601e6f32704fed600914fd5198c24395c4dc36f5' took 25.63 secs

--------------------------------------- the Error inforamtion -----------------------------------

Does anyone give me some guidance? 

Thanks

在 2015年1月30日星期五 UTC-5下午9:23:31，aab112...@hotmail.com写道：

[danny...@gmail.com]

Did you resolve the problem? I have encounted the same problem, could you give me some advices?

在 2015年2月12日星期四 UTC+8上午5:13:37，lif...@gmail.com写道：

[Ed Sutter]

Based on how often that stuff was changing when I was working with it, I seriously doubt that the

change I made will even make sense anymore... 

[Andriy Buchynskyy]

Hello, I'm trying to run 

fetch --nohooks webrtc

After about 30 minutes of running this command I receive error that similar to your. 

[0:35:29] Still working on:

[0:35:29]   src/third_party/catapult

[0:35:29]   src/third_party/ffmpeg

[0:35:29]   src/third_party/icu

Syncing projects: 100% (33/33), done.                      

Error: Command 'cipd ensure -log-level error -root /home/buch/WebRTCBuild/webrtc-checkouts -ensure-file /tmp/tmp_ooUcC.ensure' returned non-zero exit status 1

Traceback (most recent call last):

  File "/home/buch/WebRTCBuild/depot_tools/fetch.py", line 306, in <module>

    sys.exit(main())

  File "/home/buch/WebRTCBuild/depot_tools/fetch.py", line 301, in main

    return run(options, spec, root)

  File "/home/buch/WebRTCBuild/depot_tools/fetch.py", line 295, in run

    return checkout.init()

  File "/home/buch/WebRTCBuild/depot_tools/fetch.py", line 137, in init

    self.run_gclient(*sync_cmd)

  File "/home/buch/WebRTCBuild/depot_tools/fetch.py", line 82, in run_gclient

    return self.run(cmd_prefix + cmd, **kwargs)

  File "/home/buch/WebRTCBuild/depot_tools/fetch.py", line 71, in run

    subprocess.check_call(cmd, **kwargs)

  File "/usr/lib/python2.7/subprocess.py", line 511, in check_call

    raise CalledProcessError(retcode, cmd)

subprocess.CalledProcessError: Command '('gclient', 'sync', '--nohooks', '--with_branch_heads')' returned non-zero exit status 1

Can anyone please help me with this problem?