Change history implementation

[Neil Hodgson]

   Change history is the common IDE feature of displaying a coloured bar in the margin to indicate which lines have been changed. There are different coloured bars for saved and unsaved changes.

   I have implemented this for Scintilla and there is a page discussing the feature and its implementation at

https://scintilla.org/ChangeHistory.html

   The prototype is available from

https://www.scintilla.org/scite.zip  Source
https://www.scintilla.org/wscite.zip Windows executable (64-bit)

   There may be bugs and crashes. To demonstrate, visual styles were hard-coded. The code will be reorganized before its committed.

   As this may change significantly, it has not been committed to hg yet.

   Neil

[Philippe Lhoste]

This in an interesting feature, quite sophisticated. It goes a bit
beyond what I see in my daily IDE, Visual Studio Code.
I think it relies more on the VCS diffs, and doesn't make distinctions
between changes before and after saves: what counts is what changed with
the version on the repository.
Your implementation offers a vision of the undo history as well, which
can be useful or confusing… But I suppose it is up to the client of
Scintilla to choose what to show.

For the record, here is what VSC shows:
(ChangeIndicators - static)
The deleted lines are marked with a red triangle on the margin.

And when we click on one indicator, we have the following view, which is
just a diff view:
(ChangeIndicators - clic on one indicator)

For your information…
Thanks for your dedicated work on this awesome component.

--
Philippe Lhoste
-- (near) Paris -- France
-- http://PhiLhoSoft.github.io/about/
-- -- -- -- -- -- -- -- -- -- -- -- -- --

[Neil Hodgson]

Change history is the common IDE feature of displaying a coloured bar in the margin to indicate which lines have been changed.

This has been committed to Scintilla and enabled in SciTE. The 'prehistory' aspect mentioned in the initial code isn't exposed although the underlying data structures still support it so there can be further experimentation to refine the design.

Original proposal:
https://scintilla.org/ChangeHistory.html

Applications may choose whether to enable change history and whether to show the history in the margin or inline or both. Change history depends on undo history so can only be enabled when the undo history is enabled and empty.

SciTE currently turns on both margin and inline history. This may be too much and it could be changed to just show in the margin before release.

Some new visual elements were added to better present change history: SC_MARK_BAR and INDIC_POINT_TOP. INDIC_POINT, INDIC_POINTCHARACTER, and INDIC_COMPOSITIONTHICK were tweaked. The default colours and shapes for change history were changed.

https://scintilla.org/ChangeHistory.png

SC_MARK_BAR is an outlined and filled rectangle that takes the full height of the line and 1/3 of the margin width. To give a connected appearance, it displays even on wrapped lines and draws end caps on the first and last line.

INDIC_POINT_TOP is the same as INDIC_POINT but draws at the top of the line instead of the bottom.

INDIC_POINT and INDIC_POINTCHARACTER were tweaked to be 1 pixel taller and 2 pixels wider in two-phase draw mode to be clearer.

The translucency of INDIC_COMPOSITIONTHICK can be changed with SCI_INDICSETOUTLINEALPHA.

Applications may override the default appearance and SciTE provides properties to do so.

To enable this feature in a basic way in an application, call SCI_SETCHANGEHISTORY(3) after enabling undo collection after loading files.

Available fro the Hg repositories and from

https://www.scintilla.org/scite.zip Source
https://www.scintilla.org/wscite.zip Windows executable (64-bit)

Neil

[seasoned_geek]

Neil,

Minor nit.

tilla-csscintilla/src/ChangeHistory.cxx.o -c /home/roland/sf_projects/csscintilla-csscintilla/src/ChangeHistory.cxx
/home/roland/sf_projects/csscintilla-csscintilla/src/ChangeHistory.cxx:48:2: error: extra ‘;’ [-Werror=pedantic]
   48 | };
      |  ^
cc1plus: all warnings being treated as errors
[22/40] Building CXX object src/CMakeFiles/CsScintilla.dir/home/roland/sf_projects/csscintilla-csscintilla/src/Document.cxx.o

Due to my working a lot in medical device world I always compile with pretty strict options.

[Neil Hodgson]

An issue that has appeared with change markers is that SciTE's Revert command leads to the whole document getting a green change mark as revert just reloads each byte of the file. Since its useful to undo to before the revert, this point can't just be set as the start of change history to appear clear.

It may be possible to add a 'revert' state to the change history that could appear clear but its not obvious what the semantics of this state would be.

Running a diff between the on-disk file and in-memory document and only changing the ranges that differ would be possible and useful as the incoming changes would be visible. That is a bit complex so, for now, I've implemented a simpler version that finds the starting and ending ranges that are the same and replaces the range between them. This results in reasonable results in many circumstances.

This is implemented in SciTE and only works for moderate length (1MB) files in 8-bit encodings as it reads the whole file into a temporary buffer and then checks and modifies. A streaming version could be written that would handle larger files. Scintilla could also add an 'Update' API that could minimize change markers.

[Roland Hughes]

I rolled the previous changes into my editor and tested some yesterday.

This exact same problem hits my editor (RedDiamond) when styling is enabled.

Due to differing licenses I opted not to use the astyle library. Source
code formatting is performed on-save (vs. on-typing which can be
wretchedly annoying). Prior to save the file is written to a temporary,
astyle is run in-place on the temp, then the temp is reloaded before
save. Every time I save I get the solid green bar.

I don't get any of the "in text" coloring but I suspect that is because
I'm loading themes and need to provide color elements for those values.

May post a rambling a bit later this morning or today about how to fix
all of this. Just a high level "here's how the sausage is made" sketch.

Roland

--
Roland Hughes, President
Logikal Solutions
(630)-205-1593 (cell)
http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com

[seasoned_geek]

Okay, the caffeine hasn't fully kicked in, but this rattled around in my skull while I was out for my 4 mile walk yesterday and it won't leave until it comes out my fingers at a keyboard.

Admittedly Scintilla currently knows nothing about the file. We almost all have something along these lines in our code.

        m_editWidget->setText(file.readAll().constData());
        file.close();

        m_editWidget->setUndoCollection(true);
        m_editWidget->emptyUndoBuffer();
        m_editWidget->setSavePoint();
        m_editWidget->gotoPos(0);
        m_isDirty     = false;

In order to fix this, "history" problem and satisfy access to undo a bit of work and thought needs to go into the best way to make Scintilla aware of the current full file path and to adopt the JOUrnal file concept of EDT I mentioned in another thread.

Thinking out loud without thinking through here.

We need a SET_JOURNAL_LOCATION( int code, sptr_t optionalPath)

0 = current file with .jou or -jou appended

1 = User $HOME/.journal-files (most every OS has the concept of $HOME now

2 = use optionalPath - must be valid and writeable

Options 1 & 2 require name mangling like I've done in RedDiamond

https://sourceforge.net/projects/reddiamond/

so main.cpp of project A doesn't walk on the main.cpp journal of project B

All existing undo/redo code would need to be jettisoned, except for the hot keys.

At least two tables would need to exist in the SQLite (or whatever) database. Some thumbnail sketches.

CREATE TABLE DONE_TABLE( DONE_TS  TIMESTAMP  DEFAULT CURRENT_TIMESTAMP,

                         DONE_SEQ INTEGER  DEFAULT 0,

                         TYPE_CD  INTEGER,

                         UNDONE_TS TIMESTAMP DEFAULT 0,

                         UNDONE_SEQ INTEGER DEFAULT 0,

                         REPEAT INTEGER DEFAULT 0,

                         /*  whatever other fields, probably begin/end sptr_t*/

                         DOC_CHUNK BLOB,

                         PRIMARY KEY(DONE_TS, DONE_SEQ) );

                                          

CREATE TABLE UNDONE_TABLE ( UNDONE_TS   TIMESTAMP DEFAULT CURRENT_TIMESTAMP,

                            UNDONE_SEQ  INTEGER DEFAULT 0,

                            TYPE_CD  INTEGER,

                            DONE_TS TIMESTAMP DEFAULT 0,

                            DONE_SEQ INTEGER DEFAULT 0,

                            REPEAT   INTEGER DEFAULT 0,

                            /*  whatever other fields, probably begin/end sptr_t */

                            DOC_CHUNK BLOB,

                            PRIMARY KEY(UNDONE_TS, UNDONE_SEQ) );

The SEQ is needed because the timestamp on the x86 is kind of sad. Assuming individual keystrokes are being recorded, someone with a hot repeat key could write quite a few records within the same timestamp resolution.

At application start and via timer at just after midnight, current date is checked against date of most recent record in each table. If different SEQ reset to 1 for next added record. Otherwise, 1 to the highest sequence and keep going.

Exact content of the tables is not really the issue here. I know this probably isn't enough or close to it. The framework is what is important.

All freshly done stuff is in the DONE_TABLE. It's UNDONE_TS and UNDONE_SEQ is zero.

All freshly undone stuff is in the UNDONE_TABLE. The REPEAT integer is zero

When they REDO an Undone it goes back into the DONE_TABLE but now has UNDONE_TS and UNDONE_SEQ with non-zero values and the REPEAT value from the UDONE_TABLE. It also has a new TS and SEQ.

When they undo a REDO the record goes back into the UNDONE_TABLE with new TS and SEQ and an incremented REPEAT.

This lets the highlighting ignore full text replacements caused by Astyle, etc. because they can have different TYPE_CD. A few simple selects can determine UNDONE needing one highlight, DONE needing another highlight, REDONE, etc.

The external file removes the need for providing any API to expose undo history.

Clearing the history is a matter of deleting the rows in the tables. Undo and redo operations could be further restricted by TYPE_CD if needed.

The only sticky wicket is communicating to Scintilla the where for the JOUrnal file and trapping the possibility that "same location as source" is write protected directory. Perhaps it should always write to $HOME/.journal-files? Perhaps it should always demand a writeable journal path at start?

Just my 0.0002 cents.

[Neil Hodgson]

seasoned_geek:

> We need a SET_JOURNAL_LOCATION( int code, sptr_t optionalPath)

Scintilla does not do file I/O, that is always the responsibility of the application. It removes the need to handle filesystem weirdness and reduces security concerns.

> At least two tables would need to exist in the SQLite (or whatever) database.

Scintilla strongly avoids dependencies. A database dependency would be particularly onerous.

Neil