Roadblocks for Saros IDE cross-compatibility

[Tobias Bouschen]

Hi all,

I am currently in the early stages of working on IDE cross-compatibility
for Saros. For that purpose, I am looking to compile a list of roadblock
on the way to providing such cross-compatibility.

So far, I have identified the following three areas:

1. The handling of modules/project.
Suggested solution: Reference point implementation to replace project
sharing.

2. The handling of editor contents.
Suggested solution: Editor content bridge to deal with normalized
editor conent.

3. The handling of binary files.
Suggested solution: Binary flag in IFile and separate handling for
binary files (e.g. simply ignoring them).


The different points and planned solution approaches are explained in
more detail below.

I would appreciate to hear your input/feedback on this topic.
Do you see any issues with the stated roadblocks or suggested solution
approaches?
Can you think of any roadblocks/topics I might have missed?

@Stefan Rossbach
As you have good general knowledge of the Saros/E implementation and its
nitty-gritty details (and I am mainly coming at this from the viewpoint
of Saros/I), I would be very interested in having a more detailed
discussion with you on further possible internal differences between
Saros/I and Saros/E that could lead to issues.

Would you be available (and in the mood) for a Skype call (or similar)
to discuss this topic?

Best regards,
Tobias

-----------------------------------------------------------------------

1. The handling of modules/project.

This is by far the largest roadblock (requiring the largest amount of
work) I have identified so far.

The concepts of Eclipse project and IntelliJ modules are far to
different to allow for a clean mapping. This especially becomes an issue
when trying to create a module as part of the project negotiation or
when working with more esoteric module setups (e.g. weird content roots,
custom languages, etc.). Even with relatively normal setups (like the
Saros project itself), a mapping between Eclipse projects and IntelliJ
modules is far from trivial:
Eclipse sees each root "project" (e.g. Core, Eclipse, IntelliJ) as one
project.
IntelliJ, on the other hand, imports every "project" (e.g. Core,
Eclipse, IntelliJ) through Gradle as three modules, each containing
different parts of the tree (a base module containing everything but no
mapping for "source" roots, a "main" module containing the source roots
and resources, and a "test" module containing the test roots and
resources). Furthermore, some of these module contain multiple content
roots, meaning such modules can't be represented as a unified tree
(without including parents outside the module).

All in all, trying to manage IDE concepts works when only connecting to
implementations for the same IDE as we can then add very IDE-concept
specific handling logic (which is what has been done while implementing
Saros/E and Saros/I so far).
Trying to map these concepts between IDEs is far to complex to maintain,
even for "normal"/"simple" project setups.

An alternative that was already suggested in the thesis of Oliver Hanßen
is to use the concept of "reference points". This would change the Saros
infrastructure to simply share resource trees based on a reference point
in the filesystem instead of IDE-determined units of content (e.g.
projects/modules).

This would mean that we would loose some more tightly integrated IDE
features (such as creating and registering a project/module as part of
the negotiation) but would (in the end) simplify the resource handling
logic and make the adoption of new Saros implementations far easier.

My current plan is to have another stab at trying to implement such a
large scale refactoring to move the Saros filesystem to a reference
point based system.


2. The handling of editor contents.

In IntelliJ, all editor contents are normalized to use UTF-8 and UNIX
line endings.

As this is not the case for other IDEs (like Eclipse), I will need to
implement a bridge that is able to convert between normalized and
non-normalized content. Where this bridge is located, i.e. whether we
want to always deal with normalized or non-normalized content in the
general logic, has to be further discussed when I am actively working on
this roadblock.


3. The handling of binary files.

IntelliJ IDEA does not open editors for files it does not have a file
type mapping for. This is important as the normalization and subsequent
writing back to the file could damage it when opening binary files in an
IntelliJ text editor. Therefore, a distinction between binary and
non-binary files should be introduced to the core filesystem interface
to better deal with such issues. In general, I would suggest that binary
files are ignored by Saros. As far as I can tell, this is already
largely the approach taken in Saros/E (as "shared" images, for example,
are ignored in the session; they are seen as "not having an editor
selected" when in follow mode).

[Tobias Bouschen]

Hi all,

in preparation of the larger refactoring work on the filesystem API, I
would like to remove the "partial sharing" feature.

Currently, this is a very hidden feature that breaks when it is used in
more complicated setups (see partial sharing related issues:
https://github.com/saros-project/saros/issues?utf8=%E2%9C%93&q=is%3Aissue+%22partial+sharing%22).

With the current logic (excluding partial sharing), a similar result can
already be achieved using "excluded/derived" resources, as such
resources are already ignored by Saros. Furthermore, with the new logic
sharing only reference points, a similar feature set to "partial
sharing" is given by default, making it completely unnecessary in my
opinion.

All in all, removing the partial sharing feature should simplify the
codebase and make the upcoming refactoring easier.

But that is only my current view on the topic. If anybody has a
different opinion, please say so!

Otherwise, I will proceed with working on the removal in the next couple
of days (and I would like to avoid investing time into work that later
on turns out to be unnecessary/not wanted).

Best regards,
Tobias

[Tobias Bouschen]

As an addendum to the initial announcement/feedback request:

As I am working on this topic as part of my master's thesis, the
information given in the email is also present in the following forum
entry (in English as well):
https://www.mi.fu-berlin.de/w/SE/SarosCrossIDE
So have a look if you are interested in the project.

It currently contains a more detailed explanation of the project and the
preliminary set of road blocks mentioned in the email. I will also
update the entry consistently throughout my work.

Maybe I will move the content to GitHub to better integrate it with the
open source project in the future, but I am not sure about that yet.

On 1/16/20 6:56 PM, Tobias Bouschen wrote: