Pragmatic Issues

[Onorio Catenacci]

Hi all,

I'd like some advice.  One of the issues I've run into repeatedly on Windows is the issue of long directory names/directory names with spaces.  I've been working on a solution for this (to my mind the simplest way to deal with it is to find the 8.3 name of the directory).  The thing is this seems to come up so often that I think that I should make it a standalone library so that anyone who needs to deal with Windows will have it available.  But I could also see some justification for adding this (and similar issues) into the core Path module because, again, this seems to come up all the time.  

I am planning to make this a standalone library and publish on Hex.pm but I did also want to see if anyone thinks there's a case to be made to add this code into the Path module.  I'm really hesitant to suggest adding more to the main library because it's just more bloat for those who don't need it.  But I can also envision a lot of us who try to work on Windows using this over and over again. 

I'd appreciate any thoughts anyone cares to share.

--

Onorio Catenacci
 
http://onor.io

http://www.google.com/+OnorioCatenacci

[José Valim]

Onorio, can you detail those issues? In theory, the Path library was not supposed to have issues with paths with spaces on them. Such issues would happen however, when shelling out and executing scripts. For example, I know we had bugs in Mix when invoking rebar and rebar was in a directory with spaces in the name.

The thing, however, is that those issues would happen on both Unix and Windows, so I am not sure if makes sense to fix it in one of them.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAP%3DvNq8di6T82Lbyi2-L9Ba-jKqG49vrsxcO6R_PyzatbPtvgw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Onorio Catenacci]

Thanks José.

I've mainly been dealing with issues with paths with spaces when I was helping Paul with EXRM.  Mainly though, as you say most of the issues seem to come up with shelling out and running a command.

For example, if I attempt to find the Erlang path on a machine by default Erlang gets installed to "c:/program files/erl7.x" (in the latest revision, erl7.3).  If you attempt to run a command with that path (e. g. /program files/erl7.3/bin/erl), you'll get an error because of the space in program files.  This will work correctly though /"program files"/erl7.3/erl as will this /progra~1/erl7.3/erl  (NB: progra~1 is the 8.3 version of program files on my Windows machine--it may not be the same on all Windows machines due to the presence or absence of other directories that start with "program ")

This issue keeps coming up for me over and over again because almost every Windows batch file needs to check the version of Erlang which is installed which is usually accomplished by running a command from the erl command prompt. And inevitably they get something like "/program files/erl7.x/bin" for the path and it always fails.  Now the answer that seems to be the standard way of handling this is to surround the path with double quotes.  But then one can get into issues with what needs to be quoted and what doesn't. Hence I usually opt to get the 8.3 version of the name so I don't have to worry about insuring the double quotes are all in the right places. 

As I say, I'm not sure that it belongs in Path (or indeed in any standard library) and I'm perfectly happy to make this a standalone library.  On the other hand if this does seem to be something which needs to be added to path I'm happy to work on that too.

My thought is that if we were to add it to Path, it'd be something like get_short_path or something along those lines.  I would need help in figuring out how to get a short path on Linux/Unix and MacOSX because I'm not as familiar with those as I am with Windows. I'm planning to put the code I've got so far up on Github later today if you (or others) would like to look it over and see what you think.

--

Onorio

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Ka0PpB5KNBdPuMU%2By63%3Dxnd-0ii8TRDL1MnP0PX4aL1g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Onorio Catenacci]

Hi José,

Just so we're discussing actual code--my repo is here:

https://github.com/OnorioCatenacci/pragmatic

I think this might make most sense as a separate library because I don't yet have a fix for the *nix OS's.  

--

Onorio

On Mon, Mar 21, 2016 at 8:19 AM, José Valim <jose....@plataformatec.com.br> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Ka0PpB5KNBdPuMU%2By63%3Dxnd-0ii8TRDL1MnP0PX4aL1g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Jim Freeze]

Can you just quote the whole path, as in:

   "C:/Program Files/erl7.3/bin/erl"

or, escape the space, as in:

  C:/Program\ Files/erl7.3/bin/erl

Jim

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAP%3DvNq-syiY9pz03HcEcHD_k9Ze6gGj41gLcxCaeNyWVf_rgqQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Dr. Jim Freeze, Ph.D.

(m) 512 949 9683

[Onorio Catenacci]

Hi Jim,

If only things were that simple, I'd be laughing.

Just quoting the string has caused lots of problems in both the batch files for EXRM and other shell script stuff I've worked on with Windows.  Especially if the command arguments themselves ever need quotes. 

When Jose said "For example, I know we had bugs in Mix when invoking rebar and rebar was in a directory with spaces in the name."--that's been exactly my experience with EXRM.  So there are two simple answers.  Quote everything--good luck if you forget and miss a directory. Or (and this is usually the advice they give on Unix as far as I understand) avoid spaces in directory names. As I say, quotes around everything becomes a bigger issue when you take into account that some command lines want double quotes in them and then you need to escape the double quotes.

Now this would be all well and good except that Erlang defaults to installing itself in either "\Program Files" or "\Program Files (x86)" (for 32 bit versions) on Windows.  In fact I'm pretty sure that's Microsoft's standards for where they want things installed.

I can't name you a specific issue right this moment but trust me I ran into plenty of problems getting the EXRM batch files working on Windows and they took the "just quote the string" approach.  Getting the 8.3 version of the path is a lot more reliable and far less error prone.  I know that putting double quotes around the string should work correctly--that's why I called the library "pragmatic".  While simply quoting the directories should work in theory in practice it's just not that simple.  I really do wish that it were that simple.

--

Onorio

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAP1mN_z8%3DBbY%3DY_OHW2k7s404q4C75x9gCqECSnG2FyxgKKQ8w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Onorio Catenacci]

Because there are probably other folks who may think I've not examined this issue in sufficient depth consider this code:

:: Set the ERTS dir from erl

:set_erts_dir_from_erl

@for /f "delims=" %%i in ('where erl') do @(

  set erl=%%i

)

@set dir_cmd="%erl%" -noshell -eval "io:format(\"~s\", [filename:nativename(code:root_dir())])." -s init stop

@for /f %%i in ('%%dir_cmd%%') do @(

  set erl_root=%%i

)

@set erts_dir=%erl_root%\erts-%erts_vsn%

@set rootdir=%erl_root%

@goto :eof

And here's the relevant environment variables after I run this:

C:\Users\ocatenacci>set erts_dir

erts_dir=c:\Program\erts-

C:\Users\ocatenacci>set erl_root

erl_root=c:\Program

C:\Users\ocatenacci>set rootdir

rootdir=c:\Program

You'll note that because erl_root isn't set right the other environment variables are also wrong.  So the simple answer would seem to be on the line set erl_root=%%i make it set erl_root="%%i" instead.  That would seem to be the simple answer but it would be wrong.  Here's the result after I make that modification:

C:\Users\ocatenacci>set erts_dir

erts_dir="c:\Program"\erts-

C:\Users\ocatenacci>set erl_root

erl_root="c:\Program"

C:\Users\ocatenacci>set rootdir

rootdir="c:\Program"

Mind you that snippet of batch file I put in my e-mail is from an older version of the EXRM source.  It's not as if Paul and I haven't struggled to fix this and get it working right.  It's just that quoting those directories is error prone and it seems to be hard to get right.  I've been over and over this question and I'm convinced that the only consistent, practical way to fix this issue is to convert all the directories to the 8.3 version.  That's why I took the time to create that Pragmatic code--because I'm tired of trying to figure out what to quote where.

--

Onorio

[John W Higgins]

On Mon, Mar 21, 2016 at 10:59 AM, Onorio Catenacci <Cate...@ieee.org> wrote:

:set_erts_dir_from_erl

@for /f "delims=" %%i in ('where erl') do @(

  set erl=%%i

)

@set dir_cmd="%erl%" -noshell -eval "io:format(\"~s\", [filename:nativename(code:root_dir())])." -s init stop

@for /f %%i in ('%%dir_cmd%%') do @(

  set erl_root=%%i

)

Um - notice the difference between the 2 @for calls - that's your issue. You leave the delimiters as space and tab and that means it strips everything after the first space.

John

[Onorio Catenacci]

Hi John,

Thanks for that.  You may have noticed that I said "Mind you that snippet of batch file I put in my e-mail is from an older version of the EXRM source."  I wasn't presenting that code as code that I currently have an issue with.  I was presenting that code as an illustration of the general nature of the issue.

--

Onorio

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAPhAwGy%2BgSuJwaU7YvpR9VX8nPtNkW0EBSMOy7b638ec8_V8tg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Onorio Catenacci]

Your point does illustrate another issue with this though.

Making the change you suggest, I get this:

C:\Users\ocatenacci>set erl_root

erl_root="c:\Program Files\erl7.1"

C:\Users\ocatenacci>set erts_dir

erts_dir="c:\Program Files\erl7.1"\erts-

C:\Users\ocatenacci>set rootdir

rootdir="c:\Program Files\erl7.1"

So you're partially right.  Of course I didn't bother to set erts-vsn in my illustration code but using your suggestion brings up a rather interesting parsing issue.  How do I proceed with erts_dir (assuming my erts version is 7.1) in this scenario?  

erts-dir =""c:\Program Files\erl7.1"\erts-7.1"

erts-dir="c:\Program Files\erl7.1\"erts-7.1

Now, that second form probably looks fine--it may be fine in fact. Not sure; hadn't tried it. But remember this is just one isolated small sample of code.

Of course I could try to build code to parse the string, figure out if I need to strip the quotes and then re-add them.  But that hardly seems any simpler than simply working with the 8.3 versions of the paths in the first place.   Mind you that's just an isolated example.  

My point is (and was) that while adding quotes around the string sounds very simple, in practice it presents a lot of issues that can simply be avoided by using 8.3 names for paths.

On Mon, Mar 21, 2016 at 2:39 PM, John W Higgins <wis...@gmail.com> wrote:

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAPhAwGy%2BgSuJwaU7YvpR9VX8nPtNkW0EBSMOy7b638ec8_V8tg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[John W Higgins]

On Mon, Mar 21, 2016 at 10:59 AM, Onorio Catenacci <Cate...@ieee.org> wrote:

:: Set the ERTS dir from erl

:set_erts_dir_from_erl

@for /f "delims=" %%i in ('where erl') do @(

  set erl=%%i

)

@set dir_cmd="%erl%" -noshell -eval "io:format(\"~s\", [filename:nativename(code:root_dir())])." -s init stop

@for /f %%i in ('%%dir_cmd%%') do @(

  set erl_root=%%i

)

@set erts_dir=%erl_root%\erts-%erts_vsn%

@set rootdir=%erl_root%

@goto :eof

Clearly dir_cmd works fine - follow that example are you are good to go - don't add quotes unless you need to - which is when you actually use the item in a place that needs the quotes. You are adding quotes where you don't need to.

John

[Onorio Catenacci]

I believe you're missing the point I'm trying to make. 

--

Onorio

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAPhAwGyWKwaMcQ0TEyWpJSxpSoArvT3mJeb2jjtwP6%3Dt6ccZmg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[John W Higgins]

Quoting the last complaint.....

<quote>

So you're partially right.  Of course I didn't bother to set erts-vsn in my illustration code but using your suggestion brings up a rather interesting parsing issue.  How do I proceed with erts_dir (assuming my erts version is 7.1) in this scenario?  

erts-dir =""c:\Program Files\erl7.1"\erts-7.1"

erts-dir="c:\Program Files\erl7.1\"erts-7.1"

</quote>

erts_dir=%erl_root%\erts-%erts_vsn%

As long as you don't add quotes you are fine. 

 set erl=%%i

@set dir_cmd="%erl%" -noshell -eval "io:format(\"~s\", [filename:nativename(code:root_dir())])." -s init stop

That clearly works fine - because you end up with erl_root as the correct value - so the quotes around erl_root are not needed (as they were not around erl). You quote when you use the variable in a situation that requires them. Otherwise you leave it alone.

Same conceptual idea as rounding a number - you don't round until you need to otherwise you end up in a mess - don't quote until you need it.

John

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAP%3DvNq-sWxzJYvWJEmGFuL3n8o11UTy8x1J1_af1XvKk6Fga%2Bg%40mail.gmail.com.

[Onorio Catenacci]

Notwithstanding John's points about the details of the sample code I posted, I do believe there's still an area for improvement here.  I may be the only one here who uses Windows on a regular basis but I can assure you that the directory names with spaces use case is part of a larger issue and simply putting quotes around everything is not a sufficient answer. When I have a bit more time I think I can work up a far more compelling case. 

It's also worth noting that certain MS standards mandate installing in the \Program Files or \Program Files (x86) directories so I don't think this problem will go away soon either.  "Some of Microsoft’s software certification programs (such as the Windows Logo) require that applications set their default installation location to the Program Files directory." -- Raymond Chen, MSFT https://blogs.msdn.microsoft.com/oldnewthing/20120307-00/?p=8153 

However given that I promised to take some time and investigate some other stuff I will leave this discussion for another time.

For now if I get time after I attend to these investigations that I've promised, I'll put Pragmatic on Hex and then if anyone cares to use it they're free to.

--

Onorio

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAPhAwGxb-aPNiZxp4fphi9KQeWhymJR-YnC5FMta8wVKhpEpOQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.