Client build on Windows vs Linux/Mac

[Frank Bennett]

I've run into a weird snag building the Juris-M client on Windows, and as far as I can tell the issue isn't related to JM code.

When "node ./scripts/build.js" is run, the ./build directory is created and populated on all three platforms, but in my Windows 10 environment, the contents of the ./resource/bluebird subdirectory are not copied in. The problem traces back to a difference in the behavior of the module "globby," called ./scripts/js.js to populate the matchingJSFiles variable.

Here is a test script:

#!/usr/bin/node

var globby = require('globby');

async function TryGlobby() {

var matchingFiles = await globby(

[

'{chrome,components,defaults,resource,resource/web-library,test,test/resource/chai,test/resource/chai-as-promised,test/resource/mocha}/**/*.js',

'!{styles,translators,test/tests/data}/**/*.js'

],

Object.assign({cwd: '.'}),

{

"ignore": [

"**/#*.*"

]

}

);

console.log(JSON.stringify(matchingFiles, null, 2))

}

TryGlobby();

The globby version is 6.1.0 on both the Linux and the Windows systems.

The node version on Linux is 7.10.0. The node version on Windows 10 is 8.2.1.

Perhaps globby is a little stricter under the later node version, and resource/bluebird should be included in the source pattern fed to globby for it to fetch the contents of the directory?

Frank

[Dan Stillman]

On 8/3/17 1:34 AM, Frank Bennett wrote:
> I've run into a weird snag building the Juris-M client on Windows, and
> as far as I can tell the issue isn't related to JM code.
>
> When "node ./scripts/build.js" is run, the ./build directory is
> created and populated on all three platforms, but in my Windows 10
> environment, the contents of the ./resource/bluebird subdirectory are
> not copied in. The problem traces back to a difference in the behavior
> of the module "globby," called ./scripts/js.js to populate the
> matchingJSFiles variable.

You need proper symlink support on Windows. (resource/bluebird is a
symlink.) As far as I know this works with a current Cygwin install,
including Git through Cygwin, via the Cygwin terminal. Don't use an
external Git install (though there are probably settings for that that
affect this) or the Git bash shell.

[Frank Bennett]

The build is done inside cygwin64, and the git was installed from cygwin. But node and npm were installed externally (via Windows installer), and that does seem to be where the problem lies -- node w/globby under Windows, invoked inside the cygwin terminal, is not following symlinks.

Just to be sure, I downgraded the node installation in Windows 10 to 7.10.0 (to match the Linux environment), and I still get the same result -- it skips the symlinked bluebird directory.

Are the distributed Zotero installers for Windows build on native Windows, or are they cross-compiled under Mac OS or Linux?

Frank

[Dan Stillman]

On 8/3/17 9:08 AM, Frank Bennett wrote:
> The build is done inside cygwin64, and the git was installed from
> cygwin. But node and npm were installed externally (via Windows
> installer), and that does seem to be where the problem lies -- node
> w/globby under Windows, invoked inside the cygwin terminal, is not
> following symlinks.

You should also make sure you have Node 8.

> Are the distributed Zotero installers for Windows build on native
> Windows, or are they cross-compiled under Mac OS or Linux?

They're built on Windows. Everything installed via Cygwin.

[Frank Bennett]

It's slowly coming into focus.

I am (or was) running Node 8, so that's also as it should be.

In the source checked out via git, ./resource/bluebird is a proper symlink, but the ./build/resource/bluebird file created by the node ./scripts/build.js script is an ordinary file with content like this:

!<symlink>▒▒../node_modules/bluebird/js/release

A checkout of Zotero produces the same type of file. Is that normal?

If so, then differences in my build process for the standalone client must be the source of the failure.

Frank

[Frank Bennett]

Oh wait. There may be a simpler explanation.

When you say everything installed via Cygwin, does that include node and npm? If so, how should I go about acquiring cygwin versions of those two? (They're not in my fairly recent Cygwin install)

Frank

[Frank Bennett]

Digging a little further, globby depends on node-glob, and according to its docs, the double-star glob is meant to not match symlinked directories:

    https://github.com/isaacs/node-glob#glob-primer

So now I'm confused why this even works under Linux and Mac OS.

If it turns out I'm just stuck, I'll work out a kludge to copy the needed subdirectory content. Forward!

Frank

[Frank Bennett]

node-glob aka glob offers some options related to symlink behavior,
but I couldn't get any other them to work in the Cygwin environment
with the externally installed Node.

Adding a kludge that replaces the broken symlinks with actual copies
of the bluebird and web-library directories works to bring up a
working client, though. I'll just go with that for the present.

FB

> --
> You received this message because you are subscribed to the Google Groups
> "zotero-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to zotero-dev+...@googlegroups.com.
> To post to this group, send email to zoter...@googlegroups.com.
> Visit this group at https://groups.google.com/group/zotero-dev.
> For more options, visit https://groups.google.com/d/optout.

[Dan Stillman]

Oh, wait, I forgot — while the installers are built on each platform,
they're using a ZIP of the build directory that's built via Travis
[1][2], which avoids this problem. And I actually see the same behavior
as you on Windows. Tom, who built the new build system, may know more
about the Windows issues, so I'll point him to this thread, but for a
real cross-platform distribution a shared ZIP via CI is a pretty good
approach — most importantly, it guarantees that the actual bits we're
pushing out pass all tests. Really sorry for wasting your time on this —
but hopefully your debugging will get us closer to a fix for Windows folks.

[1] https://github.com/zotero/zotero/blob/master/.travis.yml#L40
[2]
https://github.com/zotero/zotero-standalone-build/blob/master/scripts/5.0_release_build_and_deploy#L15

[Frank Bennett]

No worries - I assumed that it was caused by my shortcuts at this end, and it's good to know the cause.