ANSI color coding not output in VSCode Git bash

[Brad Wood]

So, I'm not clear where the issue lies for this, but now that my CommandBox 4.0 snapshot is on JLine3, I've had people testing it out on Git Bash (since Jline2 broke support for that a while back).  Someone who uses VSCode on Windows 10 reported that none of the ANSI color coding is working, but instead, the escape sequences are just being output to the screen.  Stuff like:

←[37m5.2.4.37-SNAPSHOT SNAPSHOT (Velvet)  ←[0m

On my Windows 7 box, I have two versions of Git bash, and I honestly don't really understand the difference.  There's 

C:\Program Files\Git\git-bash.exe

which works perfectly fine and then 

C:\Program Files\Git\bin\bash.exe

which they tell me is what VSCode makes them use.  (Since I don't use VSCode, I don't really know what it allows).  Sure enough, the bash.exe above doesn't render the colors.  I googled for quite a while and found tons of similar tickets, many in the VSCode repo itself for very similar issues with PHP, and npm users, but all of them seemed to lack a clear solution, other than just saying that bash terminal "doesn't support color".  

Does anyone have some experience with this who can help me know what to do here?  Does my JLine-powered app need to detect some sort of flag and attempt to strip all ANSI formatting or something in certain terminals??

Thanks!

~Brad

[Brad Wood]

An update on this-- at first based on my research, I thought the bash.exe that ships with Git just didn't support ANSI, but this is not the case.  npm programs we tested that output in color work fine.  I also tested JLine2 and it outputs colors fine as well in Git's bash.exe.  So at this point, I'm unclear on what's going on.  The escape codes are being streamed out to the terminal with JLine3, but the terminal is simply outputting them and not acting on them.  What do I need to check?  

Here's what I get when starting CommandBox 3.9 which uses JLine 2.12. 

This is what I get when starting CommandBox 4.0 which uses JLine 3.5.1

I'll also note that some keys don't work in the latter scenario either like the up arrow key doesn't trigger the history stuff.  I can only assume that JLine3 isn't detecting the correct terminal but I'm not sure how to troubleshoot that.  The only other thing I've changed that might have possibly affected this is I'm now using a slightly newer version of Launch4J to generate my Windows executable, but I don't know if it has the opportunity to affect terminal detection.

Thanks!

~Brad

Developer Advocate

Ortus Solutions, Corp 

E-mail: br...@coldbox.org

ColdBox Platform: http://www.coldbox.org 

Blog: http://www.codersrevolution.com

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brad Wood]

Ooh, this looks promising.  I found this difference while testing both the git-bash.exe binary (which works) and the bash.exe binary (which doesn't work and is the default of VSCode) 

git-bash.exe

reader.getTerminal()

   org.jline.terminal.impl.PosixSysTerminal

reader.getTerminal().getType()

   xterm-256color

bash.exe

reader.getTerminal()

   org.jline.terminal.impl.PosixSysTerminal

reader.getTerminal().getType()

   mcygwin

You can see that both terminals are using PosixSysTerminal, however git-bash.exe is a type of xterm-256color (this is the one that shows colors properly and stuff like up arrow works for history) and the bash.exe shows a terminal type of mcygwin (the broken one).  

What controls the terminal type and how do we tell what it should be?  The Terminal class in JLine2 doesn't have a getType() so I don't know what type is being reported in JLine2, but I do know what it shows up as jline.DefaultTerminal2.

Thanks!

~Brad

Developer Advocate

Ortus Solutions, Corp 

E-mail: br...@coldbox.org

ColdBox Platform: http://www.coldbox.org 

Blog: http://www.codersrevolution.com

[Guillaume Nodet]

Troubleshooting the terminal creation can be done if you can put a breakpoint in TerminalBuilder#doBuild() method and debug from there.

Whatever the terminal type is mcygwin, the jline PosixSysTerminal will only work correctly if infocmp returns a valid information for that terminal type.

If that's not the case, a custom information can be build by adding a .caps file at the following location:

  https://github.com/jline/jline3/tree/master/terminal/src/main/resources/org/jline/utils

and referencing it in the following method:

  https://github.com/jline/jline3/blob/master/terminal/src/main/java/org/jline/utils/InfoCmp.java#L617-L620

[Brad Wood]

When I run infocmp from the Git bash.exe I get the following.  Can you tell me if that's valid or not?

Also, can you confirm if JLine3 works for you in the bash.exe that ships with the Git client for Windows?  I'm curious what other JLine users are doing who have an app used inside of VSCode.  

Brad@development MINGW64 /bin

$ infocmp

#       Reconstructed via infocmp from file: /usr/share/terminfo/63/cygwin

cygwin|ansi emulation for Cygwin,

        am, hs, mir, msgr, xon,

        colors#8, it#8, pairs#64,

        acsc=+\020\,\021-\030.^Y0\333`\004a\261f\370g\361h\260j\331k\277l\332m\300n\305o~p\304q\304r\304s_t\303u\264v\301w\302x\263y\363z\362{\343|\33

0}\234~\376,

        bel=^G, bold=\E[1m, clear=\E[H\E[J, cr=^M, cub=\E[%p1%dD,

        cub1=^H, cud=\E[%p1%dB, cud1=\E[B, cuf=\E[%p1%dC,

        cuf1=\E[C, cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA,

        cuu1=\E[A, dch=\E[%p1%dP, dch1=\E[P, dl=\E[%p1%dM,

        dl1=\E[M, ed=\E[J, el=\E[K, el1=\E[1K, fsl=^G, home=\E[H,

        hpa=\E[%i%p1%dG, ht=^I, ich=\E[%p1%d@, ich1=\E[@,

        il=\E[%p1%dL, il1=\E[L, ind=^J, invis=\E[8m, kb2=\E[G,

        kbs=^H, kcub1=\E[D, kcud1=\E[B, kcuf1=\E[C, kcuu1=\E[A,

        kdch1=\E[3~, kend=\E[4~, kf1=\E[[A, kf10=\E[21~,

        kf11=\E[23~, kf12=\E[24~, kf13=\E[25~, kf14=\E[26~,

        kf15=\E[28~, kf16=\E[29~, kf17=\E[31~, kf18=\E[32~,

        kf19=\E[33~, kf2=\E[[B, kf20=\E[34~, kf3=\E[[C, kf4=\E[[D,

        kf5=\E[[E, kf6=\E[17~, kf7=\E[18~, kf8=\E[19~, kf9=\E[20~,

        khome=\E[1~, kich1=\E[2~, knp=\E[6~, kpp=\E[5~, kspd=^Z,

        nel=^M^J, op=\E[39;49m, rc=\E8, rev=\E[7m, ri=\EM,

        rmacs=\E[10m, rmcup=\E[2J\E[?47l\E8, rmir=\E[4l,

        rmpch=\E[10m, rmso=\E[27m, rmul=\E[24m, rs1=\Ec\E]R,

        sc=\E7, setab=\E[4%p1%dm, setaf=\E[3%p1%dm,

        sgr=\E[0;10%?%p1%t;7%;%?%p2%t;4%;%?%p3%t;7%;%?%p6%t;1%;%?%p7%t;8%;%?%p9%t;11%;m,

        sgr0=\E[0;10m, smacs=\E[11m, smcup=\E7\E[?47h,

        smir=\E[4h, smpch=\E[11m, smso=\E[7m, smul=\E[4m, tsl=\E];,

        u6=\E[%i%d;%dR, u7=\E[6n, u8=\E[?6c, u9=\E[c,

        vpa=\E[%i%p1%dd,

Thanks!

~Brad

Developer Advocate

Ortus Solutions, Corp 

E-mail: br...@coldbox.org

ColdBox Platform: http://www.coldbox.org 

Blog: http://www.codersrevolution.com

[Guillaume Nodet]

Could you try building and using the following branch:

  https://github.com/gnodet/jline3/tree/gitwinfix

It seems to work better for me.

[Brad Wood]

Sure, I'll give it a shot.  I've been playing around with the debugger for the last hour or so, but since I don't know what any of these variables are supposed to be, it's hard for me to tell what's wrong.  I do know that OSUtils.IS_CYGWIN is false and OSUtils.IS_MINGW is true.  i also know that the "type" is "xterm" in git-bash.exe and it's "cygwin" in bash.exe so the if statement that was on line 283 didn't override the type, but when I modified the environment variables such that the type was always xterm-256color, that didn't seem to have any effect.  I've traced through the constructor for PosixSysTerminal and I see everything running, but I'm not sure what it means.  I do know what the output of infocmp is different between git-bash.exe and bash.exe.

Thanks!

~Brad

Developer Advocate

Ortus Solutions, Corp 

E-mail: br...@coldbox.org

ColdBox Platform: http://www.coldbox.org 

Blog: http://www.codersrevolution.com

[Brad Wood]

Ok, great news-- your latest build seems to work now on both git-bash.exe and bash.exe!  It skips the entire 

 if ((OSUtils.IS_CYGWIN || OSUtils.IS_MINGW) && exec)

block and now creates an instance of "JnaWinSysTerminal" with a type of "windows".  

I am curious if that's really correct now that IS_MINGW returns false though since the actual terminal window shows "MINGW64" in the title bar.  I don't quite understand the difference between cygwin and mingw, nor do I know why the TERM env var returns "cygwin" when the title bar has the word "MINGW" in it.  However, it does seem as though the terminal should actually be cygwin or mingw.  Is it correct that both of those are false now?

Thanks!

~Brad

Developer Advocate

Ortus Solutions, Corp 

E-mail: br...@coldbox.org

ColdBox Platform: http://www.coldbox.org 

Blog: http://www.codersrevolution.com

[Brad Wood]

I had my VSCode user do some testing and he confirmed that build make color work for him.  However, I'm still curious if your fix is really the correct one since the shell seems to clearly be advertizing that it's mingw, yet we're detecting it as NOT mingw with this fix.  Will this break something else?

Thanks!

~Brad

Developer Advocate

Ortus Solutions, Corp 

E-mail: br...@coldbox.org

ColdBox Platform: http://www.coldbox.org 

Blog: http://www.codersrevolution.com

[Guillaume Nodet]

I think this is ok.  The reason is that there are 2 different terminals in cygwin / mingw.  When the TERM returns "cygwin", we need to use the JLine an AbstractWindowsTerminal derived implementation (using jansi or jna) whereas if that's not the case (TERM is usually "xterm"), we can use the PosixSysTerminal.

I'll change the code to make things more clear.

[Brad Wood]

Ok, sounds good then. I'd say merge this into the main project!

When do you expect the next stable release to be? I'm looking to have a final release of CommandBox 4.0 prior to my company's conference in April.  It would be great to get all of these little improvements in by then.

[Guillaume Nodet]

When you stop raising issues... ;-)

I'm working on #140 right now.  If I can have something working (and you stop raising issues...), I'll do a release, so definitely before April I hope.

[Brad Wood]

Heh, sorry to cause so much work.  You can see why I put this off for over a year!  I'm super glad to have made the jump to JLine3 tho, the syntax highlighting and completor stuff is really going to be a nice improvement.  I'm also working on a BulletTrain module for CommandBox that will look sweet.  Not specific to JLine3 per se, but part of my next major release.

Thanks!

~Brad

Developer Advocate

Ortus Solutions, Corp 

E-mail: br...@coldbox.org

ColdBox Platform: http://www.coldbox.org 

Blog: http://www.codersrevolution.com