Text "left over" in console after clearing screen

214 views
Skip to first unread message

Brad Wood

unread,
Jan 4, 2018, 12:24:48 AM1/4/18
to jline...@googlegroups.com
There's a behavior I've noticed for a quite a while but I always assumed it was just Windows cmd being stupid.  However, I've been playing with ConEMU and Hyper today which are totally different terminals, but they are showing the exact same behavior.  

The problem is that after I issue a clear screen:

reader.clearScreen();
reader.getTerminal().writer().flush();

As I keep working in the shell, once I've produced more than one screen worth of output (meaning I've reached the bottom and the text has started scrolling off the top), I start to see old "cleared" text scrolling past that is peeking out from the ends of the new lines being printed.  

I can reproduce this very easily by starting up CommandBox and running the "info" command a number of times which produces a bunch of ASCII art over and over. Then I run the "CLS" command which clears the screen.  Finally, if I just hold down the enter key to keep redrawing the prompt, as the new prompts stream by, I'll see old text that I had previously cleared off the screen.  It's like the lines are being reused and the new text is being drawn on top of them but the old lines were never actually cleared.

At first glance, I'm unable to reproduce this on JLine2.  I've also checked through all the source code and there are no places where I write directly to the output stream myself.  All content to the shell is being output via

reader.getTerminal().writer().print( arguments.string );
reader.getTerminal().writer().flush();

Here's a screenshot that shows 3 different layers of output.  The prompt is being drawn over and over on the left.  Peeking out from that in the middle is some ASCII art from previous commands before I cleared the screen.  And even longer lines are peeking out on the right from some error messages on the screen back before I output the ASCII art.  

Inline image 1

Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 

Guillaume Nodet

unread,
Jan 4, 2018, 4:23:17 AM1/4/18
to jline...@googlegroups.com
I think the clearscreen will only erase the visible area and not the full window.

It may be interesting if you could test with the following instead:
  
COORD topLeft = new COORD();
topLeft.x = 0;
topLeft.y = 0;
int screenLength = info.size.y * info.size.x;
FillConsoleOutputAttribute(console, originalColors, screenLength, topLeft, written);
FillConsoleOutputCharacterW(console, ' ', screenLength, topLeft, written);

Also, you need to modify the jna related code in the same way:

If it works for you, feel free to create a pull request to fix the problem :-)


--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Brad Wood

unread,
Jan 4, 2018, 9:41:00 AM1/4/18
to jline...@googlegroups.com
> I think the clearscreen will only erase the visible area and not the full window.

Yikes, I usually have a buffer size of 5,000 lines in my console just so I can scroll back really far.  That's a lot of lines not getting cleared!

> It may be interesting if you could test with the following instead:

I can give it a try.  I'm curious how many Windows users there are using Jline-powered projects.  Some of this stuff is a little surprising to be yet-uncovered.  

Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Guillaume Nodet

unread,
Jan 4, 2018, 11:09:42 AM1/4/18
to jline...@googlegroups.com
On Thu, Jan 4, 2018 at 3:40 PM, Brad Wood <bdw...@gmail.com> wrote:
> I think the clearscreen will only erase the visible area and not the full window.

Yikes, I usually have a buffer size of 5,000 lines in my console just so I can scroll back really far.  That's a lot of lines not getting cleared!

They are not supposed to be visible.  I'm not sure why in your case they end up showing again.
 

> It may be interesting if you could test with the following instead:

I can give it a try.  I'm curious how many Windows users there are using Jline-powered projects.  Some of this stuff is a little surprising to be yet-uncovered.  

I don't think this happen in the usual case.  Are you sure you don't do anything fancy with the screen / window console ?

Brad Wood

unread,
Jan 4, 2018, 11:14:28 AM1/4/18
to jline...@googlegroups.com
> They are not supposed to be visible.  I'm not sure why in your case they end up showing again.

Well, it makes sense if the clear screen moves the cursor back to the first line of the terminal.  After I add enough lines to scroll down, I'm going past lines I've already been past before and if they're not getting cleared, then it makes sense that I'd see what's still on them!

> Are you sure you don't do anything fancy with the screen / window console ?

Nope, nothing that I'm aware of.  Where is another JLine powered app I can grab to play with?  It's very easy to reproduce in CommandBox with just a basic Windows cmd window.  The only thing I've ever touched is increasing the terminal width and height so it fills more of my screen.

Inline image 1


Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Guillaume Nodet

unread,
Jan 4, 2018, 11:47:58 AM1/4/18
to jline...@googlegroups.com
On Thu, Jan 4, 2018 at 5:14 PM, Brad Wood <bdw...@gmail.com> wrote:
> They are not supposed to be visible.  I'm not sure why in your case they end up showing again.

Well, it makes sense if the clear screen moves the cursor back to the first line of the terminal.  After I add enough lines to scroll down, I'm going past lines I've already been past before and if they're not getting cleared, then it makes sense that I'd see what's still on them!

Does it ?  I think it should move the cursor to the first visible line, not to the real first line.  That may be the root of the problem and could be a better fix instead of clearing thousands of lines for nothing.
I think the processCursorTo may need to offset the line so that there's no scroll hapenning

Brad Wood

unread,
Jan 4, 2018, 11:52:46 AM1/4/18
to jline...@googlegroups.com
> I think it should move the cursor to the first visible line, not to the real first line.  

Does it really matter though if it's the very first line or just the first visible line?  If there's not-visible lines below the current viewport that have text still on them, they're still going to come into view as soon as I add enough lines to the current screen to start scrolling down again.  Unless you're saying that the lines "below" the current viewport would somehow always be empty.

Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Guillaume Nodet

unread,
Jan 4, 2018, 12:00:58 PM1/4/18
to jline...@googlegroups.com
On Thu, Jan 4, 2018 at 5:52 PM, Brad Wood <bdw...@gmail.com> wrote:
> I think it should move the cursor to the first visible line, not to the real first line.  

Does it really matter though if it's the very first line or just the first visible line?  If there's not-visible lines below the current viewport that have text still on them, they're still going to come into view as soon as I add enough lines to the current screen to start scrolling down again.  Unless you're saying that the lines "below" the current viewport would somehow always be empty.

Yes, I think that's the expectation.  At least, I've always assumed that scrolling down is sufficient to bring a new empty line.
Given JLine should not offer any way to go back in the buffer (or will not, if the cursor positioning is fixed), it should work.

At the end, the behavior should be the same than hitting Ctrl+L in the git-bash shell, or cls in the windows cmd shell.

Guillaume Nodet

unread,
Jan 4, 2018, 12:26:49 PM1/4/18
to jline...@googlegroups.com
Actually, it seems that the raw windows cmd actually go back to the first line while other cygwin/mingw terminals do erase the viewport and not the whole buffer (same as unix terminals afaik).
So I'd rather default to the second option, eventually making that configurable if there's a need for it.
Brad, could you try fixing the cursor positioning to end up with a correct behavior ?

Cheers,
Guillaume

Brad Wood

unread,
Jan 4, 2018, 12:30:23 PM1/4/18
to jline...@googlegroups.com
> So I'd rather default to the second option,

I'm down for whatever option makes clear screen actually work :)

> Brad, could you try fixing the cursor positioning to end up with a correct behavior ?

Are you referencing the code you suggested I try in WindowsAnsiWriter?  It's on my list to try today after I'm done fiddling around with the Git statuses in CommandBox bullet train.

Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Brad Wood

unread,
Jan 5, 2018, 11:06:13 PM1/5/18
to jline...@googlegroups.com
I tried putting your code in both WindowsAnsiWriter classes, but it made no change.   I attached a debugger and those functions are never even called!  The clearScreen() method I'm calling on the line reader simply runs 

terminal.puts(Capability.clear_screen)

which eventually runs 

Curses.tputs(writer(), str, params);

Are the methods you mentioned supposed to be running?

My terminal is org.jline.terminal.impl.jna.win.JnaWinSysTerminal

Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Brad Wood

unread,
Jan 6, 2018, 1:04:31 AM1/6/18
to jline...@googlegroups.com
Oops, I didn't step into the right methods.  The processEraseScreen() method was getting called, but some debugging code I had put in the first case statement didn't run because the ERASE_SCREEN_TO_END was the case getting hit and not ERASE_SCREEN.  Should erase to the end be the one getting hit when calling reader.clearScreen()?

The str being passed to tputs is \E[H\E[J and processEraseScreen is what's defaulting to the ERASE_SCREEN_TO_END option (0).


Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Brad Wood

unread,
Jan 6, 2018, 1:30:01 AM1/6/18
to jline...@googlegroups.com
Ok, if I make the changes you suggested of changing the coords so the entire terminal is cleared AND I update the ANSIWriter to default to option 2 for case J like so:

                case 'J':
                    processEraseScreen(optionInt(options, 0, 2));
                    return true;

Then I actually seems to get the screen completely cleared!  I no longer can get the text from previous output to show up. 

So I think the question is whether defaulting the option int for processEraseScreen should have been ERASE_SCREEN along.  If you don't see any issues with that, I'll try sending over a pull!

Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Brad Wood

unread,
Jan 6, 2018, 1:52:04 AM1/6/18
to jline...@googlegroups.com
I went ahead and made a pull just in case.  


Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Guillaume Nodet

unread,
Jan 6, 2018, 3:32:48 AM1/6/18
to jline...@googlegroups.com
I've googled a bit more, and I'd rather make the behavior consistent.
The "erase screen" options do work correctly.  Some unix terminals include support for erasing the scrollback using the value 3 for the erase screen option.
We could implement this option if needed, but in the mean time, I'd rather fix the cursor positioning with the following (untested) patch.
This alone should work and make the following erase_screen(0) work correctly (instead of having to call erase_screen(2) instead).


diff --git a/terminal-jansi/src/main/java/org/jline/terminal/impl/jansi/win/WindowsAnsiWriter.java b/terminal-jansi/src/main/java/org/jline/terminal/impl/jansi/win/WindowsAnsiWriter.java

index c3f0196..37fe9fd 100644

--- a/terminal-jansi/src/main/java/org/jline/terminal/impl/jansi/win/WindowsAnsiWriter.java

+++ b/terminal-jansi/src/main/java/org/jline/terminal/impl/jansi/win/WindowsAnsiWriter.java

@@ -241,7 +241,7 @@ public final class WindowsAnsiWriter extends AnsiWriter {

     @Override

     protected void processCursorTo(int row, int col) throws IOException {

         getConsoleInfo();

-        info.cursorPosition.y = (short) (row - 1);

+        info.cursorPosition.y = (short) (info.window.top + row - 1);

         info.cursorPosition.x = (short) (col - 1);

         applyCursorPosition();

     }

diff --git a/terminal-jna/src/main/java/org/jline/terminal/impl/jna/win/WindowsAnsiWriter.java b/terminal-jna/src/main/java/org/jline/terminal/impl/jna/win/WindowsAnsiWriter.java

index 0ce3c06..383b45f 100644

--- a/terminal-jna/src/main/java/org/jline/terminal/impl/jna/win/WindowsAnsiWriter.java

+++ b/terminal-jna/src/main/java/org/jline/terminal/impl/jna/win/WindowsAnsiWriter.java

@@ -231,7 +231,7 @@ public final class WindowsAnsiWriter extends AnsiWriter {

 

     protected void processCursorTo(int row, int col) throws IOException {

         getConsoleInfo();

-        info.dwCursorPosition.Y = (short) (row - 1);

+        info.dwCursorPosition.Y = (short) (info.srWindow.Top + row - 1);

         info.dwCursorPosition.X = (short) (col - 1);

         applyCursorPosition();

     }


Brad Wood

unread,
Jan 6, 2018, 2:35:10 PM1/6/18
to jline...@googlegroups.com
I reverted my other changes and tried your example above.  It does appear to work in a way.  It clears one screen-worth of the buffer and leaves nothing but empty terminal below.  If I open a shell and fill a few lines with content and then "cls", then basically everything goes away because there wasn't more than one screen worth of content.  

However, if I generate two windows worth of lines and then run "cls" then it leaves one screen worth of buffer at the top and the last window-worth gets cleared.  That means if I scroll up, I see cut off content above.  So long as that is expected, then I'd say we're good to go.  Honestly, I'm not sure if there is a spec for how a clear screen should work and if that spec (should it exist) says what should happen to any text above the current view port.  

Here's the new pull:


Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Brad Wood

unread,
Jan 6, 2018, 2:53:07 PM1/6/18
to jline...@googlegroups.com
Nevermind on the pull.  When you said untested, I thought that meant you hadn't committed it yet. I see you already added that commit.

Thanks!

~Brad

Developer Advocate
Ortus Solutions, Corp 

ColdBox Platform: http://www.coldbox.org 


Guillaume Nodet

unread,
Jan 7, 2018, 2:17:01 AM1/7/18
to jline...@googlegroups.com
Yeah, I tested it in the mean time.
The behavior seems consistent with other terminals.
We can add something else to clear the whole buffer if that’s needed.

Guillaume


To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jline-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jline-users...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages