gclient failing

[Meraj Jilani]

Hi,

I am trying to build chromium for windows. when I am running gclient command in command prompt (admin mode) it gives error :

Failed to bootstrap or update CIPD client
Downloading CIPD client for windows-amd64 from https://chrome-infra-packages.ap
pspot.com/client?platform=windows-amd64&version=git_revision:0323cbe1ef467af36a
a6784f2315c5ee36e89e34...
Failed to download the file, check your network connection
At E:\depot_tools\.cipd_impl.ps1:116 char:5
+     throw "Failed to download the file, check your network connection"
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Failed to downl...work connec
   tion:String) [], RuntimeException
    + FullyQualifiedErrorId : Failed to download the file, check your network
   connection

Failed to bootstrap or update CIPD client

Any idea what is going wrong here?

Thanks,

Meraj

[PhistucK]

Any chance you are behind a proxy?

If so, I would try to set environmental variables called HTTP_PROXY and HTTPS_PROXY to the URL of the proxy.

☆PhistucK

--
--
Chromium Discussion mailing list: chromium...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-discuss

---
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-discu...@chromium.org.

[Christian Biesinger]

FWIW, it's probably a bad idea to do this in admin mode (and possibly
(?) the reason why it's failing)

Christian

> --
> --
> Chromium Developers mailing list: chromi...@chromium.org

> View archives, change email options, or unsubscribe:

> http://groups.google.com/a/chromium.org/group/chromium-dev
> ---
> You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
> To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAAVD3YAJTA3Hu%2BMZT6Vd5zEtbNh-Pm_GAFCPSd9dOTy80CpUxg%40mail.gmail.com.

[Meraj Jilani]

Hi, 

I have tried below and none of them work.

1: launch command prompt in admin mode and hit gclinet.

2. launch command prompt in non admin mode and hit gclient.

3: since I am behind the corporate proxy so I have set environment variable HTTPPROXY pointing to corporate proxy URL(with port).

In all cases getting the same error. My internet is working fine.

Has anyone ever faced this issue? Please assist.

Thanks,

Meraj

[PhistucK]

You mentioned (privately and not) "http_proxy" and "HTTPPROXY".

I suggested "HTTP_PROXY" and "HTTPS_PROXY" (I think "HTTPS_PROXY" is even more relevant), so, just noting in case that is the culprit.

☆PhistucK

--

--
Chromium Discussion mailing list: chromium...@chromium.org

View archives, change email options, or unsubscribe:

http://groups.google.com/a/chromium.org/group/chromium-discuss

---
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-discu...@chromium.org.

[Meraj Jilani]

Hi, 

I can download cipd.exe from the browser but it's failing through gclient command.

I have tried setting environment variables http_proxy (and HTTP_PROXY) and https_proxy(HTTPS_PROXY)) but still getting the same error message.

Does gclient really use these environments variables to check proxy settings?

Thanks,

Meraj

[Meraj Jilani]

After more digging, I found that while downloading the file it's throwing the exception "could not create SSL/TLS secure channel".

Any idea how to fix this? Does it need some specific version of .Net framework?

Thanks,

Meraj

[Meraj Jilani]

Okay, it works once I added code to support tls -1.2 in "depot_tools\.cipd_impl.ps1"

but now gclient sync giving below error:

 for match in self.ANSI_OSC_RE.finditer(text):
TypeError: cannot use a string pattern on a bytes-like object

Any idea what's going wrong now?

Thanks,

Meraj

[Bruce Dawson]

What OS are you running on? There was a similar (but not identical) error reported a few times when trying to get a Chromium repo on Windows 7. In that case the trick was to redirect the gclient output to a file, but I'm not sure this will help here.

[Meraj Jilani]

Hi,

I am using Windows 8.1, gclient command has installed two versions of python, 2.7 and 3.8.

I noticed on chromium discussion forum that this issue could be due to an incompatible version of python also.

But currently in my environment variable path list depot_tool  is placed at first place and there is no other version of python pre-installed.

Thanks,

Meraj

[Bruce Dawson]

We'll need to see more of the output in order to understand what is going on - all of it, ideally, perhaps as a text-file attachment if it is too unwieldy to paste into the email thread.

Also, you said:

Okay, it works once I added code to support tls -1.2 in "depot_tools\.cipd_impl.ps1"

It would be nice if you could share the diffs or attach the modified file so that we can understand what change you made and whether we need to incorporate it. Ideally developers should be able to follow the steps and get the code without having to patch it and it would be helpful to understand what is going wrong. Windows 8.1 tends to get light usage these days so it is quite possible that a bug in its support has snuck in and if so we would want to fix it.

chromium-discuss@ removed due to bounced messages.

--

Bruce Dawson

[Meraj Jilani]

Hi,

Please find attached cipd_impl.txt (extension modified), line number 114 has my changes to enable tls-1.2.

Without adding this, it was throwing the exception "could not create SSL/TLS secure channel".

After the above modification gclient works fine and it downloaded python version 2.7.17 and python3.bat (version 3.8) in depot_tool folder.

But then the command "gclient sync --with_branch_heads"  starts failing with TypeError: cannot use a string pattern on a bytes-like object.

This error was certainly due to python version, then I found some old version of depot_tool which has python version 2.7.6, and

I added environment variable DEPOT_TOOLS_UPDATE = 0, after this "gclient  gclient sync --with_branch_heads" starts working and downloading chromium code but looks like it could not download chromium completely and throws error (error.png attached).

Why I am saying that chromium is not completely downloaded because I could not see gn.exe inside the folder "src\buildtools\win".

Shall I move to windows 10?

Trying to clone chromium on my machine since the last 10 days but no luck, any help would be highly appreciated.

Thanks,

Meraj

[Bruce Dawson]

Thank you for the information. I'm not a networking expert so I don't know the implications of the change to .cipd_impl.ps1, shown below, and whether it is expected that it might be needed on Windows 8.1. Anybody have thoughts?

  $wc = (New-Object System.Net.WebClient)
  $wc.Headers.Add("User-Agent", $UserAgent)
  try {
    [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12
    $wc.DownloadFile($URL, $TmpPath)
  } catch {

    $err = $_.Exception.Message
    throw "Failed to download the file, check your network connection, $err"
  }

Off-hand I think your fix looks reasonable, although I don't know what it implies that it is necessary. I'll see if the owners think it is appropriate.

Your next fix is not safe. By pinning yourself to an old version of depot_tools you have pinned an old version of gclient_eval.py and that is what caused your latest problem. You can see in this change when the 'Var is the only allowed function' message was removed and replaced with a more permissive check. Chromium depends on that more permissive check. So, you need to get rid of DEPOT_TOOLS_UPDATE = 0 and then we can figure out why gclient sync is failing.

In order to understand that problem we will need the full output, including the call stack. The output attached to your latest email had the information necessary but it is much more convenient to have it as text rather than as a .png. The Windows command prompt does let you select text, although I think it might be off by default. See the edit, defaults, and properties options in the command prompt system menu. The more output the better, for understanding what is happening.

Thank you for reporting these issues.

--

Bruce Dawson

[Meraj Jilani]

Thanks Bruce,

I have enabled DEPOT_TOOLS_UPDATE and now using the latest version of depot_tool to download chromium.

Actually gclient is default using python 3 to download chromium and python 3 has some known issues while dealing with utf-8 string

and gives error TypeError: cannot use a string pattern on a bytes-like object.

I have added environment variable GCLIENT_PY3 = 0 to force gclient to use python 2 and by doing so everything works fine and the chromium codebase downloaded successfully.

Suggestion: Since python 3 has some issues, so chromium's default behavior should be to use python 2.

I am attaching two files here success.txt (when python 2 was enabled) and error.txt (when python 3 was enabled).

In case of python 2 (success.txt), though chromium was downloaded successfully but it's showing some NOTICE: for https_proxy.

Can you check whether that notice is okay or I am missing something? and chromium not downloaded correctly. Chromium is throwing this notice because

I have added the https_proxy environment variable. 

Thanks,

Meraj

[Takuto Ikuta]

Could you file an issue in https://bugs.chromium.org/p/chromium/issues/entry ?

I think an issue with the Infra>SDK component gets attention from appropriate owners.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAAVD3YC03rsOgb%2BohSQH09gNzGspsPXwB3j66j9Lscftmk8Q%2Bw%40mail.gmail.com.

[Dirk Pranke]

Hi Meraj,

At the moment, I'm not aware of any issues with Python 3 and gclient, and I'd expect it to work, so that's why it's the default; we'd rather not move that back to Py 2 unless we have to.

It's quite possible that there are bugs, of course, as tikuta@ suggests. We'll look into them as quickly as we can.

I encourage everyone who is hitting issues with gclient to file bugs, Python 3-related or otherwise. We can't fix stuff we don't know about :).

-- Dirk

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CALNjmMo4aLbXEbWcSZBU-%3DH%3DhamR%2BzN07HVdmhHxpzP-aHex2A%40mail.gmail.com.

[Bruce Dawson]

BTW, I uploaded the change to handle TLS 1.2 to crrev.com/c/2480605 if anyone who knows the implications better wants to take a look (or if the original poster wants to patch it in to test it)

--

Bruce Dawson

[Bruce Dawson]

It looks like the "TypeError: cannot use a string pattern on a bytes-like object" failure is a colorama/Python 3 issue that has been filed a few times. From reports I've seen elsewhere I think that the issue is that it fails on Windows 7 (and now, apparently, Windows 8.1) which is why we can't reproduce it.

The bug has been reported at least twice. I deduped so crbug.com/1114548 is the master now.

--

Bruce Dawson

[Dirk Pranke]

Don't we effectively require Win 10 to build now? I wonder if we should be hard-failing on earlier OS'es.

(And I wonder if gclient still needs to support earlier OS'es even if Chromium doesn't ...).

-- Dirk

[Bruce Dawson]

I think that Chrome still builds fine on earlier OS versions (except for this bug). And we still run on Windows 7.

I think that we could easily fix this (don't use colorama on < Windows 10) and then all would be well, at least for now. If we hit any more serious obstacles then we could block earlier operating systems but I don't think we need to at the moment.

--

Bruce Dawson

[Meraj Jilani]

Hi,

Any idea about the download throwing NOTICE for https_proxy? Please see attached success.txt file, to me it seems chromium downloaded successfully but not sure due to those notices.

Thanks,

Meraj

[Bruce Dawson]

Those just look like warnings. You'd have to experiment with the suggested changes to try to make them go away, or just ignore them.

--

Bruce Dawson

[Dirk Pranke]

Okay. Blocking colorama on < Win10 seems perfectly acceptable to me.

-- Dirk