Yarn Install Issue

[wiki...@gmail.com]

File generation with

yarn install

finalizing with error

error An unexpected error occurred: "https://github.com/ryankshaw/jquery.git: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?".

It seems there is issue not in yarn, but something is wrong with the github repository. Please, advise.

[Jorge Herskovic]

I had this. It's an issue in yarn; it asks github to return an octet-stream, which github refuses to do. 

I believe it was fixed here: https://github.com/yarnpkg/yarn/pull/3987 

In the meantime, until the patch makes its way downstream (and not ideal, I admit) I removed the 'Accept': 'application/octet-stream' line from my yarn installation in /usr/share/yarn/lib/yarn-cli.js which is part of this call:

    return registry.request(this.reference, {
      headers: {
        'Accept-Encoding': 'gzip',
        Accept: 'application/octet-stream'
      },

This solved the problem for me.

Best 

Jorge

[wiki...@gmail.com]

Hi, Jorge!

Thank you so much for problem outlined!

Newest version of Yarn v1.0.2 has different paths and codes. The similar code can be found in /usr/share/yarn/lib/cli.js, see it: https://drive.google.com/open?id=0B6BQheZ4N_OsY1QtcVRjVnRxaE0

The file cli.js contains no 'Accept': 'application/octet-stream' header:

return yield registry.request(_this3.reference, {

            headers: {

              'Accept-Encoding': 'gzip'

            },

What I know about the problem. The 'yarn install' returns the error:

yarn install

yarn install v1.0.2

$ script/gem_npm install

running yarn install for gems/canvas_i18nliner/package.json

yarn install v1.0.2

[1/4] Resolving packages...

success Already up-to-date.

Done in 0.43s.

running yarn install for gems/selinimum/package.json

yarn install v1.0.2

[1/4] Resolving packages...

success Already up-to-date.

Done in 0.32s.

[1/5] Validating package.json...

[2/5] Resolving packages...

[3/5] Fetching packages...

warning Pattern ["jquery@https://github.com/ryankshaw/jquery.git#1.7.2-with-AMD-and-CommonJS"] is trying to unpack in the same destination "/home/plus/.cache/yarn/v1/npm-jquery-1.7.2-a755a3e9c99d5a70d8ea570836f94ae1ba56046d" as pattern ["jquery@>=1.7.1","jquery@>=1.5","jquery@>=1.2.6"]. This could result in a non deterministic behavior, skipping.

error An unexpected error occurred: "https://github.com/ryankshaw/jquery.git: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?".

info If you think this is a bug, please open a bug report with the information provided in "/var/www/lms/yarn-error.log".

Yarn log file contains the error info:

Trace: 

  Error: https://github.com/ryankshaw/jquery.git: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?

      at Object.module.exports.exports.decode (/usr/share/yarn/lib/cli.js:69569:43)

      at Extract.onheader (/usr/share/yarn/lib/cli.js:128814:39)

      at Extract.module.exports.Extract._write (/usr/share/yarn/lib/cli.js:128937:8)

      at doWrite (/usr/share/yarn/lib/cli.js:66232:64)

      at writeOrBuffer (/usr/share/yarn/lib/cli.js:66221:5)

      at Extract.module.exports.Writable.write (/usr/share/yarn/lib/cli.js:66158:11)

      at Duplexify.ondata (/usr/share/yarn/lib/cli.js:65306:20)

      at emitOne (events.js:96:13)

      at Duplexify.emit (events.js:188:7)

      at readableAddChunk (/usr/share/yarn/lib/cli.js:64977:18)

Canvas LMS file package.json contans 2 github repositories, which can't be fetched by Yarn returning same 'tar error' (see above). The repositories are:

https://github.com/ryankshaw/fullcalendar.git#aa686b36d10cee1e1e3ec7c7784145e46667d47d
and
https://github.com/ryankshaw/jquery.git#1.7.2-with-AMD-and-CommonJS

I have tried:

curl -I -H 'Accept: application/octet-stream' -H 'Accept-Encoding: gzip' https://codeload.github.com/ryankshaw/fullcalendar/tar.gz/aa686b36d10cee1e1e3ec7c7784145e46667d47d

It gives the response:

HTTP/1.1 200 OK

Content-Length: 646478

Access-Control-Allow-Origin: https://render.githubusercontent.com

Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'

Strict-Transport-Security: max-age=31536000

Vary: Authorization,Accept-Encoding

X-Content-Type-Options: nosniff

X-Frame-Options: deny

X-XSS-Protection: 1; mode=block

ETag: "aa686b36d10cee1e1e3ec7c7784145e46667d47d"

Content-Type: application/x-gzip

Content-Disposition: attachment; filename=fullcalendar-aa686b36d10cee1e1e3ec7c7784145e46667d47d.tar.gz

X-Geo-Block-List: 

Date: Sat, 16 Sep 2017 07:54:00 GMT

X-GitHub-Request-Id: E335:27A5D:63453A:893072:59BCD898

NPM fetches both mentioned above repos without any problem, but Yarn returns error. You are right, this is the Yarn issue, not the repo. The only question how to resolve the issue?

пʼятниця, 15 вересня 2017 р. 22:48:00 UTC+3 користувач Jorge Herskovic написав:

[wiki...@gmail.com]

Additionally I have tried to fetch the jquery github repo directly (without package.json):

yarn add g...@github.com/ryankshaw/jquery.git

The error is same:

yarn add g...@github.com/ryankshaw/jquery.git

yarn add v1.0.2

$ script/gem_npm install

running yarn install for gems/canvas_i18nliner/package.json

yarn install v1.0.2

[1/4] Resolving packages...

success Already up-to-date.

Done in 0.43s.

running yarn install for gems/selinimum/package.json

yarn install v1.0.2

[1/4] Resolving packages...

success Already up-to-date.

Done in 0.30s.

[1/5] Validating package.json...

[2/5] Resolving packages...

Couldn't find any versions for "git" that matches "github.com/ryankshaw/jquery.git"

? Please choose a version of "git" from this list: 0.1.5

[3/5] Fetching packages...

error An unexpected error occurred: "https://github.com/ryankshaw/jquery.git: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?".

info If you think this is a bug, please open a bug report with the information provided in "/var/www/lms/yarn-error.log".

info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

пʼятниця, 15 вересня 2017 р. 20:32:13 UTC+3 користувач wiki...@gmail.com написав:

[Jorge Herskovic]

Yes, and both ryankshaw repos are the ones that error out. I was running whatever yarn you get from the yarnpkg.com repo for Debian and it had the above problem. I can confirm that it has been patched since, the filenames changed, and the latest version  does NOT have the line I asked you to remove.

So that's not the problem right now, and I don't know what is.

Another thing that might help is emptying the yarn cache, in your case: 

rm -rf /home/plus/.cache/yarn

as yarn is, to the best I can tell, INSANELY aggressive about caching stuff. You may just be bumping against the already-downloaded bad tarfile.

I'll try to confirm that yarn install works on my machines as of today... will respond soon.

Jorge

[Jorge Herskovic]

Ok, as far as I can tell I *can* reproduce the error as of a few minutes ago. FWIW, npm install seems to work at the moment. 

yarn, as much as it is faster and newer than npm, doesn't handle some edge cases as well. 

I'd love to hear from Graham or Simon if they know whether using npm is a terrible idea at this point.

[Graham Ballantyne]

npm should work, but it’s a pain in the butt since the Docker config runs yarn by default. 

When I ran into this last week, master was working fine but not stable. Also odd: our build server installed stable fine, but it was probably pulling the jquery package from yarn’s cache. 

Question for Ryan: why are you pulling in a jquey fork?

-- 

Graham Ballantyne 

IT Services 

Simon Fraser University 

--

---
You received this message because you are subscribed to the Google Groups "Canvas LMS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to canvas-lms-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jorge Herskovic]

This jquery fork is pulled by the Canvas repo (I have the same issue, verified today again). See https://github.com/instructure/canvas-lms/blob/stable/yarn.lock 

[Jorge Herskovic]

FWIW, I just updated using npm install instead of yarn and it worked fine; I regenerated the assets using 

COMPILE_ASSETS_NPM_INSTALL=0 RAILS_ENV=production bundle exec rake canvas:compile_assets

to avoid another (failed) round of installations, and this seemed to work well.

[Jorge Herskovic]

Spoke too soon; not all assets were generated correctly.

[Shawn Iverson]

Have you tried downgrading yarn to 0.27-5?

To unsubscribe from this group and stop receiving emails from it, send an email to canvas-lms-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Shawn Iverson, CETL

Director of Technology

Rush County Schools

765-932-3901 x271

iver...@rushville.k12.in.us

[Jorge Herskovic]

I saw that literally a minute ago in the issues, tried it, and it built successfully. Thanks. 

[wiki...@gmail.com]

The issue can be resolved by installing packages from fullcalendar repo separately. Assume Canvas LMS installation directory is '/var/www/canvas' and yarn version is v1.0.2. We have to do the following before 'yarn install' issuing:

canvasuser@host:/var/www/canvas$ rm -rf ~/.cache/yarn/*
canvasuser@host:/var/www/canvas$ mkdir ~/canvtmp
canvasuser@host:/var/www/canvas$ cp package.json ~/canvtmp
canvasuser@host:/var/www/canvas$ rm -rf package.json
canvasuser@host:/var/www/canvas$ yarn add g...@github.com/ryankshaw/fullcalendar.git#aa686b36d10cee1e1e3ec7c7784145e46667d47d
canvasuser@host:/var/www/canvas$ cp ~/canvtmp/package.json .
canvasuser@host:/var/www/canvas$ rm -rf ~/canvtmp
canvasuser@host:/var/www/canvas$ yarn install

During command

yarn add g...@github.com/ryankshaw/fullcalendar.git#aa686b36d10cee1e1e3ec7c7784145e46667d47d

execution select g...@0.1.5 version.

That's all. The issue is resolved.

пʼятниця, 15 вересня 2017 р. 20:32:13 UTC+3 користувач wiki...@gmail.com написав:

File generation with