j2s GC and Canvas

[Jacob Beard]

Hi,

I'm currently working on a project for Google Summer of Code for the
Eclipse Foundation. I've been trying to leverage GWT in this project,
but I think j2s might actually be a better target. I'd like to present
my ideas here and see what people thing.

The project is to take GMF, and allow it to produce web-based diagram
editors instead of SWT-based ones. My project mentor has asked that,
rather than write a custom generator that goes generates
platform-specific code for the UI (in HTML, CSS and JavaScript)
directly from the EMF generator model by writing a custom generator,
that I continue to use the current GMF Java generator, and then
compile the resultant Java code (built on the Draw2d/SWT/JRE stack) to
platform-specific JavaScript. The e4 project has been using this
strategy to target ActionScript/Flash with some success.

I've been attempting to use GWT to do this, but unfortunately, this
effort has so far not been very productive. It seems that Eclipse GC
is more deeply tied to other parts of SWT than I first realized, and
this has made porting it a much larger project than I originally
intended. Also, GWT's Eclipse tooling has been highly problematic.

It seems to me that Java2Script might be more closely aligned to the
goals of my project, as it comes with good SWT emulation, and better
Eclipse tooling. In terms of SWT emulation, the most critical things
for my project are GC and Canvas. I took a look at the source code to
GC and it looks like most of of it is stub code, except for a few
methods which seem to do simple graphical operations using DOM and CSS
styling. This is to be expected, and perhaps I can contribute code
that would allow a more sophisticated implementating of GC that would
target HTML Canvas (or Google's excanvas, for compatibility with IE).
The way I imagine doing this is simply filling in the stub methods
with JSNI native code. I'd like to know what the j2s developers think:
does it seem like a more complete implementation of GC should be
fairly straightforward to "slot in"? More generally, does it sound
like Java2Script would align well with the needs of my project?

I'd appreciate it if you would let me know what you think. Thanks,

Jake

[Zhou Renjian]

Hi Jocob,

Using HTML Canvas, SVG or IE's VML for Draw2D/SWT in Java2Script is a feasible way for web based diagram editor. But you must know that the compliance across different browsers may take you a lot of time. And you may need times to optimize the implemenation for better performance.

My early thought about Draw2D/GEF or GC/Canvas implementation is similar to your thought. But there is no spefic project driving for this kind of implementation at that time.

I think you also need to think about the performance of Java2Script SWT implementation for your project. In my opinion, GMF is a really big framework. So you may need to learn a lot of performance tuning, from lazy loading to code/pattern refactoring. Even modern browser's JavaScript engines are running very fast, you still need to take performance in mind from the start.

In all, I think Java2Script will get your work done.

If you have other questions, please let us know.

If you do choose Java2Script's SWT for you project, I recommend you to setup Java2Script environment from SVN repository and use latest sources ( net.sf.j2s.core/ui on branch_3.3 and others on trunk).

Regards,
Zhou Renjian

--
http://j2s.sourceforge.net/ Java2Script
http://webuzz.im/ Web IMs (G/A/M/Y)

[Jake B]

Great, thanks for the advice on this. I'm going to get started and
I'll let you know how it goes.

Best regards,

Jake

[Sebastian Gurin]

Hi Jacob.

I have experience in GEF but not in GMF, but nevertheless, I think yours is a great and ambitious idea. In the case of GEF, it is virtually impossible to build a visual editor outside eclipse I hope it is not the case of GMF...

I've using svg, htmlcanvas and other approaches for drawing in the browser. In this mailing list files (http://groups.google.com/group/java2script/files) there is an mathWebThings.zip (updated) which is an eclipse project with some examples of java2script apps. I think the following things can be interesting to you:

org.sgx.j2s.bitmap.html - the fastest way I found for drawing points and lines using html elements.

org.sgx.j2s.raphael - my java2script porting to the great vector drawing library raphaeljs (svg/vml)

org.sgx.j2s.bitmap.canvas - my little html canvas element java api

org.sgx.j2s.fractals.JuliaShell - julia fractal drawer using the faster way of the below list in current browser.

org.sgx.fplotter - a swt function ploter

org.sgx.AppsMenu the main application menu.

I hope can help you to make your own javascript drawing primitives.

In my experience, htmlcanvas is faster than svg/vml (but not supported by all browsers). svg/vml have a rich set of drawing primitives, it is relative well supported in all browsers, but I'm not sure if it is sufficient fast for a visual editor (specially for a VE drawing tools)

Well, good luck, and please keep us informed about your progress. and feel free to use this examples as you wish..

--
Sebastian Gurin <sgu...@softpoint.org>

[Jake B]

Hi Sebastian,

Thank you for your input! I'll definitely investigate the resources
you linked to.

I'm curious, why do you say that it is impossible to build a visual
editor outside of Eclipse using GEF? GEF is part of the stack that GMF
targets, so hopefully this challenge will not be insurmountable. What
certain members of the e4 project have done to address this is to
create a Java-to-ActionScript compiler, and create a fairly complete
emulated SWT layer, which also includes an implementation of GC. There
are advantages and disadvantages to this approach, but they have
achieved pretty good results. With a good enough Java-to-JavaScript
compiler, it seems like the same thing should be possible to do with
JavaScript. And, thanks to HTML Canvas, there should not even be too
much of an impedance mismatch between the immediate-mode GC API, and
the immediate-mode Canvas API. That was a critical consideration, as I
do not think it would be feasible to efficiently implement GC on top
of a retained-mode API like SVG (even though I really like SVG).

Unfortunately, I've been having trouble using Java2Script for this
project so far due to bugs in the j2s compiler. For example this one:

https://sourceforge.net/tracker/?func=detail&aid=2797539&group_id=155436&atid=795800

I spent some time looking into it today, but was unable to resolve it
due to my unfamiliarity with the code. I may have to go back to GWT
because I feel like their compiler might be more solid (even though I
haven't been able to compile that much :P). I'll look into it a bit
more tomorrow and over the weekend, and hopefully I'll be able get it
resolved.

Best regards,

Jake

[Jacob Beard]

Hi Sebastian,

I'm currently trying to get a sense of what will be required to
implemented GC on top of Java2Script, and so I've been looking for
reduced examples. I've been using the code you linked to as a starting
point. It looks like there are two locations where GC is referenced in
your mathWebThings project: org.sgx.j2s.widget.swt.SWTWidget, and
org.sgx.j2s.widget.swt.tests.Snippet252. Snippet252 appears to be a
reduced, straightforward example of GC, for drawing simple shapes, so
that's good. SWTWidget, on the other hand, I'm not clear on. The
intent seems to be to inject some logic that allows GC to work for
Java2Script widgets, but beyond that, I'm not sure.

I have a few questions, whenever you get a chance:
* Snippet252 worked for me as a regular Java application, but
unfortunately, the corresponding Java2Script application did not
appear to draw anything. Did you ultimately have any success with
this?
* How does Snippet252 (and applications that want to leverage GC in
general) deal with GC's reliance on the Paint event, an event which at
the moment I do not believe is exposed by any browser except Mozilla
nightlies? Does this get translated to some other browser native
event?
* Can you explain a bit the intent behind SWTWidget?
* Can you forsee any further difficulties in implementing GC on top of
Java2Script, beyond what I have described here? Any further insights
or observations?

Please let me know what you think. Thanks,

Jake

2009/5/29 Jake B <otak...@gmail.com>:

[Sebastian Gurin]

Hello Jacob. I haven't implemented the swt GC in none of my project's package. What I tried to show with the projects pointed in my last mail was how one can use html canvas, svg or pure html for drawing in the browser in a java2script project, but are not related with swt GC.

The two classes you mention that reference swt gc are not javascript translatable. SWTWidget is a part of my ambitious idea: make a little, scalable, widget toolkit for java2script, more web oriented and easyly to implement in javascript, swt, swing, etc. But it is not related to swt internals or GC...

About my thoughts about implementing swt GC in java2script, I think if you use htmlcanvas or svg you won't have troubles implementing drawing primitives. But like any other java program you will have to bypass mechanism and components not supported in the browser, like concurrence, java.io. etc

On Sun, 31 May 2009 14:23:48 -0400

--
Sebastian Gurin <sgu...@softpoint.org>

[Jacob Beard]

Hi Sebastian,

Thanks for the quick response. This certainly does put things more
into perspective.

Right now, I'm trying to figure out one of the problems that I
posed in my previous email, namely,
that the browser doesn't expose a native paint event on its native
widgets, and the simplest examples of GC involve a pattern in which
you set a listener for this event on a widget and then pull the GC off
of the paint event and use that to draw on the widget. It seems like
it will be necessary to emulate the Paint event as we don't have it
natively, but it's not yet clear to me what is the best way to do
that. I have an idea though, and I'd appreciate it if I could get your feedback.

At the moment, I think the solution is going to involve injecting some
logic into the Control constructor to trigger a paint event manually
after the Control is initialized. I think it is only receiving the
initial paint
event that is problematic, because it is generated by the underlying
system. The other way to receive the paint event is
when the user calls Control.redraw, so that should be relatively easy
to hook up, as it is initiated by the user and not the underlying
system. My questions are:

* What do you think about the above solution?
* Where would be the best place to inject the logic that manually
triggers a paint event? I was thinking it would e in the constructor
of Control.
* Can you think of any other cases where SWT widgets need to be
repainted?

Let me know what you think. Thanks,

Jake

[Sebastian Gurin]

HI jacob, first of all I think you would be more interested in Zhou Renjian's opinion since he is the main author of swt for java2script and have dealed with swt issues like you list... ;)

It is true that detecting paint events in an html dom is not possible. But, if you are considering other richer standars for GC implementation, for example svg or htmlcanvas, then you may have all what you need...

IMHO (please correct me if I'm wrong) swt java2script implementation "paint" controls relying on css properties (bounds, absolute positioning, z-index, etc) and not in a GC (for example, Widget:setBackgroundColor() method is responsible of setting css dom properties suitably). Because of this approach, onpaint events lose its relevance on swt java2script current implementation.

It is your idea to re-implement swt Controls so they are painted via a GC instead using css?


On Mon, 1 Jun 2009 11:07:16 -0400

--
Sebastian Gurin <sgu...@softpoint.org>

[Jacob Beard]

Hi Sebastian,

Replies below:

On Mon, Jun 1, 2009 at 1:32 PM, Sebastian Gurin <sgu...@softpoint.org> wrote:
> HI jacob, first of all I think you would be more interested in Zhou Renjian's opinion since he is the main author of swt for java2script and have dealed with swt issues like you list... ;)

I would love to get Zhou's input on this as well. Zhou, feel free to
chime in here: what do you think would be the best way to emulate the
paint listener for j2s?

>
> It is true that detecting paint events in an html dom is not possible. But, if you are considering other richer standars for GC implementation, for example svg or htmlcanvas, then you may have all what you need...

I just rechecked the specs for HTML5 and SVG, and I haven't been able
to find anything about a specific paint event, unfortunately, so I
don't think that this is currently a possibility.

>
> IMHO (please correct me if I'm wrong) swt java2script implementation "paint" controls relying on css properties (bounds, absolute positioning, z-index, etc) and not in a GC (for example, Widget:setBackgroundColor() method is responsible of setting css dom properties suitably). Because of this approach, onpaint events lose its relevance on swt java2script current implementation.

What I imagine is that all of the current infrastructure for drawing
SWT widgets would be left in place. However, GC would still exist so
that, if you wanted to, for example, draw paths on top of the widget,
or use any of the rich graphical API's that GC provides, you would be
able to do so. The way this would be implemented in DOM is there would
be a Canvas element with a transparent background with a width and
height equal to the Widget's bounding box, overlayed on top of the
Widget. I believe this would have the desired effect.

>
> It is your idea to re-implement swt Controls so they are painted via a GC instead using css?

I think that, at least for the first implementation, we should have both.

[Zhou Renjian]

For a button with image, and the image is drawn by GC, it is like the followings:
    /**
     * @param image
     * @param color
     */
    void drawImage (Image image, Color color) {
        GC gc = new GC(image);
        gc.setBackground(color);
        Rectangle bounds = image.getBounds();
        gc.fillRectangle(0, 0, bounds.width, bounds.height);
        gc.drawRectangle(0, 0, bounds.width - 1, bounds.height - 1);
        gc.dispose();
    }

...
        drawImage (backgroundImage, color);
        backgroundButton.setImage (backgroundImage);
...

In side Button#setImage, the GC's drawing is emulated by DIV elements. You can change it into Canvas or SVG.

And for paint event, as Jacob said it can be emulated inside redraw(). You can override it and add paint event like the following:

public void redraw() {
    super.redraw();
    this.redrawTick++;
    /**
     * @j2sNative
     * window.setTimeout((function (tf, tick) {
     *     return function () {
     *         if (tf.redrawTick == tick) {
     *             tf.onPaint (null);
     *         }
     *     };
     * }) (this, this.redrawTick), 100);
     */ {}
}

I use this above code to re-implement CTabFolder. And as it is not yet submitted, I just uploaded as file:
http://groups.google.com/group/java2script/web/ctabfolder.zip

Hope all these codes help.

Regards,
Zhou Renjian

--
http://webuzz.im/ Web IMs (G/A/M/Y)
http://j2s.sourceforge.net/ Java2Script

[Jake B]

Hi Zhou,

Thanks for the input! Your sample code worked great, and it was
extremely helpful. This is great, because I now have a simple way of
interacting with the GC API, I can start writing native code behind it
that leverages the Canvas API. Fun stuff to come!

I think it's still an open question as to how to handle Paint
listener. This will inevitably need to be dealt with, as Draw2d and
the rest of the stack make use of the pattern of setting a paint
listener on the Canvas widget. Here's a snippet from
org.eclipse.draw2d.LightweightSystem:

protected void addListeners() {
EventHandler handler = createEventHandler();
canvas.getAccessible().addAccessibleListener(handler);
canvas.getAccessible().addAccessibleControlListener(handler);
canvas.addMouseListener(handler);
canvas.addMouseMoveListener(handler);
canvas.addMouseTrackListener(handler);
canvas.addKeyListener(handler);
canvas.addTraverseListener(handler);
canvas.addFocusListener(handler);
canvas.addDisposeListener(handler);
canvas.addListener(SWT.MouseWheel, handler);

canvas.addControlListener(new ControlAdapter() {
public void controlResized(ControlEvent e) {
LightweightSystem.this.controlResized();
}
});
canvas.addListener(SWT.Paint, new Listener() {
public void handleEvent(Event e) {
LightweightSystem.this.paint(e.gc);
}
});
}

Zhou, do you have any ideas as to the best way to deal with this?
Perhaps a better question is, in the browser context, what precisely
do we expect will trigger a repaint (besides the user manually calling
repaint)? There is the initial painting that occurs sometime after the
shell is initialized; it's possible that after resizing the Canvas, it
will automatically repaint, although this behaviour can be turned off.
I'm not sure what other cases will need to be dealt with. I'd greatly
appreciate it if you could let me know if you can think of any other
situations that will trigger a repaint.

Best,

Jake

[Sebastian Gurin]

Hi Jacob, these are my opinions about your idea I give my humble opinions:

* it would be interesting to compare current swt widget implementation using html elements with widgets represented by svg or htmlcanvas based GC.

* current swt implementation advantages are that is lightweight and can be useful to represent a swt widget hierrarchy as a xml DOM elment hierarchy.

* current swt implementation "(d)efect" is that since it is not GC based, widget drawing must me done manually for each widget type. In this context I think using a GC approach has some advantages like less drawing code (this means less drawing bugs), more portable and maintanable. For example It would be easier to upgrade to a newer swt version or support other swt based toolkits like jface in a GC based SWT implementation than in current implementation.

* it would be nice if java2script user be able to configure swt renderization method (html+css or GC based) for each widget type...


On Mon, 1 Jun 2009 14:08:19 -0400

--
Sebastian Gurin <sgu...@softpoint.org>

[Jacob Beard]

Hi Sebastian,

Replies below:

On Wed, Jun 3, 2009 at 3:37 PM, Sebastian Gurin <sgu...@softpoint.org> wrote:
>
> Hi Jacob, these are my opinions about your idea I give my humble opinions:
>
> * it would be interesting to compare current swt widget implementation using html elements with widgets represented by svg or htmlcanvas based GC.

This would be interesting, however, at the moment, it doesn't really
overlap with my project goals for this summer. My main motivation is
to get the stack required by GMF to work in the browser, and so the
intent is primarily to get the SWT Canvas Widget and the corresponding
GC API to work.

>
> * current swt implementation advantages are that is lightweight and can be useful to represent a swt widget hierrarchy as a xml DOM elment hierarchy.

True, but SVG is also hierarchical and exposes an XML DOM. In any
case, I'm not sure how useful an XML DOM really is; manipulating DOM
tends to be slower than pure JavaScript, and so the question is, if
your entire abstract representation of the SWT UI was maintained as a
data structure in JS, would it be faster or slower than if maintained
that representation in HTML or SVG? For things like implementing the
event model (bubbling, capturing, etc.) I imagine that the XML
representation would be faster, as it's implemented in a faster
language, but for dynamic manipulation of that data structure, I
believe that there's a strong likelyhood that a pure-js implementation
would be faster, as it would be js manipulating pure-js data
structures.

>
> * current swt implementation "(d)efect" is that since it is not GC based, widget drawing must me done manually for each widget type. In this context I think using a GC approach has some advantages like less drawing code (this means less drawing bugs), more portable and maintanable. For example It would be easier to upgrade to a newer swt version or support other swt based toolkits like jface in a GC based SWT implementation than in current implementation.

If I understand you correctly, you would like to use a GC emulation
layer as a low-level graphical abstraction layer to replace the
hand-coded "painting" in HTML. That's actually a pretty interesting
idea. The implementation of SWT widgets in general does not use GC as
an abstraction layer, so you wouldn't be able to reuse code there, but
if you think that using GC in this way will make the j2s code more
portable and maintainable, then that sounds like it has some pretty
interesting possibilities.

Thanks for the feedback on this,

[Jacob Beard]

FYI, here is roughly how I envision GC will map onto Canvas:

http://spreadsheets.google.com/ccc?key=rE_-5E2ZMBtROmptVxoK4pw

Best,

Jake