Cygwin support

[Arthur Norman]

I see from a year back that the core team do not have time to check installation on all possible setups, specifically cygwin and myss2, and that for Windows testing they maybe look to WSL. My main work has used Cygwin fo rmany years and having WSL activated risks confusion and conflict for me, so I do not see it as a good option - and even more so if I wanted to include emscripten stuff in my main project.

When I try a fresh install on cygwin (or mysys2) I get a failure

Traceback (most recent call last):
  File "./emsdk.py", line 3055, in <module>
    sys.exit(main())
  File "./emsdk.py", line 2799, in main
    load_sdk_manifest()
  File "./emsdk.py", line 2338, in load_sdk_manifest
    add_sdk(sdk)
  File "./emsdk.py", line 1972, in add_sdk
    raise Exception('Duplicate sdk ' + str(sdk) + '! Existing:\n{' + ', '.join("%s: %s" % item for item in vars(find_sdk(str(sdk))).items()) + '}, New:\n{' + ', '.join("%s: %s" % item for item in vars(sdk).items()) + '}')
Exception: Duplicate sdk sdk-upstream-master-64bit! Existing:
{version: upstream-master, bitness: 64, uses: ['llvm-git-master-64bit', 'node-12.18.1-64bit', 'python-3.7.4-pywin32-64bit', 'emscripten-master-64bit', 'binaryen-master-64bit'], os: win, id: sdk, name: sdk-upstream-master-64bit, is_old: False, is_sdk: True}, New:
{version: upstream-master, bitness: 64, uses: ['llvm-git-master-64bit', 'node-12.18.1-64bit', 'emscripten-master-64bit', 'binaryen-master-64bit'], os: linux, id: sdk, name: sdk-upstream-master-64bit, is_old: False, is_sdk: True}

which looks as if the script is confused as between Windows and Linux (and we can all se that that is reasonable!).

So my questions

Is there experience or expectation that if I get past installation that emscripten will work under cygwin?

If not then I should avoid wating my time. But if this ia liable to be fairly shallow I can investigate further and hope to report back with any patch or fix I find. So either is there somebody with a solution already or will following up on this be a good use of my time or a rabbit-hole of unlimited depth?

Arthur

[Floh]

I bet this is because running "python" in a cygwin shell runs a Windows python installation where the platform.system() call returns "Windows", so the emsdk python scripts think that they are running in a Windows environment. 

Trying to install python in the cygwin environment doesn't seem to setup a "python" link for me, instead the exe is called python3.8.exe. Calling that returns "CYGWIN_NT-6.1-7601" in the platform.system() call, which is very different from the "usual" return values "Darwin", "Linux" and "Windows", so some cygwin-special hacks for the platform detection might be need in any case.

Not sure if that's worth it to be honest (but I can't speak for the emscripten team of course). FWIW I've been using the emscripten SDK in a "native" Windows environment just fine, there have been various small Windows-specific problems popping up in the past, but that's a few years back. From your description it sounds like emscripten is not the main reason you are running cygwin or mingw on Windows though.

[Sam Clegg]

I think this is know issue: https://github.com/emscripten-core/emsdk/issues/385

I imagine the fix could be easy, but we also want to think about the right approach for how to support such polyglot configuration. e.g.
  
windows python + running in msys2 sh.exe

msys2 python + running in cmd.exe

It gets even more complicated if you try to run an autoconf script: https://github.com/emscripten-core/emscripten/issues/12376.  Specifically, it's not clear to me how to setup the emsdk environment such that it work in all cases.  Do we set EMSDK_BLAH=c:\blah or EMSDK_BLAH=/c/blah?  I imagine msys2 has solutions to this problem but I'm not at a familiar with it  Help would be very much appreciated!

Ideally we would add testing for all this stuff to emsdk and maybe some minimal tests in emscripten.

A quick fix patch could also work if you'd like to propose one.

cheers,

sam

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/23852e32-f3c1-4bcf-a985-407ba91e388en%40googlegroups.com.

[Arthur Norman]

Thanke everybody. I have tried a range of paths. By editing emsdk.py to deletct cygwin via sys.platform=='cygwin' I can adjust the sdk compatibility tests so that the Linux one is not deemed useful (I thik that was there because "posix" got detected). That let me complete emsdk install. (well I also had to disable the hack that patches for path-names > 160 chars long, because cygwin hated the expanded name, while windows 10 since 2016 or so does not have that limitation). But then the unzip had created many files such that I had to go "chmod +x" on them before cygwin would launch them. When I did that I could achoieve emsdk activate. Then I edited a DOS batch file for setting environment variables to be a bash one (I faile dto make the script to create that in bash mode work at all). and overrode EMSDK_PYTON to use the cygwin python3.

That let me try to launch emcc and after I had fix4ed TMP so that temporary files went somewhere where both DOS and cygwin could agree that the path was valid I get failure with emcc passing a load of paths of the form /cygdrive/d/emsdk/... down to clang, while clang does not understand that idiom and would need d:/emsdk/...

I have done some diving to see where that might be patched - eg by wrapping use of abspath to create a dos-friendly result. But this the starts to get deeper into emscripten that I can sensibly afford to go.

I do not thing that the alterations I have made are liable to be tidy or significant enough to be worth passing on. They are at the sort of level that anybody used to the python code in emsdk etc would put in almost instantly and where they may know the best plavce to put them too.

So I am sad bot that this did not work out of the box on cywgin and that I have failed to find what I still suspect is a smallish set of patches. But from the stuff I do elsehere (reduce-algebra.sourceforge.net) I understand well that keeping cross platform working against all options is hugely time-consuming and often the visible payback may not justify it.

Arthur

[Sam Clegg]

I wonder if once you have installed emsdk it would make sense to use the windows version of python to run emcc?   i.e. run emcc.bat from the cygwin shell?   Rather than trying to make it cygwin all the way down.

The deeper into emscripten the cygwin changes go the more effort over time it will be to maintain this mode.

Does cygwin not have some kind of automatic translation of paths such that when I run `compiler.exe /some/cygwin/path` it can turn it into `compiler.exe c:/some/wndows/path`? 

What happens for example if you run `cl.exe /some/cygwin/path` .. does it simply not work?

Perhaps if we instead consider the windows version of emscripten to be a windows compiler (just like cl.exe)?  (I mean it literally is a windows compiler because it is based on the windows version of llvm underneath as you have discovered).

I'm beginning think think we should hard error in emscripten if `sys.platform=='cygwin'` (or the msys equivalent) because we don't support this mode at all.

cheers,

sam

To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/bd499845-af15-4d6e-ab2b-4d392bd5800bn%40googlegroups.com.

[Arthur Norman]

Typical windows code can cope with local paths and ones expressed as c:\somewhere\xxx.py etc, but not onto things lile /cygdrive/c/somewhere/... The use of "/" or "\" is accepted by a decent proportion of things. In fact I am now trying having installed the windpows-only emscripten and going

   cmd /c emcc ...

from cygwin and for simple cases that works. In the past I did similar (but worse) things to be able to launch the cygwion 32-bit toolchains from a 64-bit world or vice versa and experience with that leaves me feeling it is not a very wonderful solution long term or to use on anything I wish to distribute.

Your tests in the scruipts for mingw/msys suggests that that worked at one stage and I still feel that it looks as if the issues are really to do with interfacing tje tools to the outside world not to truly deep intsrnal issues. Eg a cygwin python tends to arrange that when yoiu call abspath to get the absolute path of the executable so you can access other things relative to it it tends to return /cygdrive/c/cywgin64/home/USERNAME/emsdk but then windows tools need that mapped onto C;/... for then to accept it - not a totally dreadul challenge to somebody whgo knows their way around the cod eenough to spot path-related code (maybe).

But yes, I fear that if it is not going to be made to work (sad face!) it would save frustration if it reported that clearly on the first attempt, and if the FAQ explained just what configurations it does behave in.

Since I run in a world where I use autoconf and a range of other Unix/Linux tools I do not use "cl" as a compiler on Windows. I use gcc/g++ for building my utilities and x86_64-w64-mingw32-c++ to build the native windows code to send out. I have little idea of the profile of your existing and potential user-base but it may well be that people liile me are unusual enough not to justify significant effort - and that can be fair enough! But the responses I have had to me questions here have been polite, prompt and helpful and that is very good indeed! Thank you all.