JsInterop retrofit

[Goktug Gokdogan]

I finally had some time to think about JsInterop and how to abstract browser APIs.

I will try to summarize my thought process and different options that I have played with so that you could get a better understanding where I'm coming from and I hope it will provide good documentation for future.

So I started by playing with the static functions, constructors and how to represent that with the current JsInterop semantics.

First I started with the original approach: use @JsType on an interface and use some simple conventions to generate the rest:

@JsType(prototype = "Object")
interface JsObject extends Auto_JsObject {

  static JsObject of(Object obj) {
    return obj instanceof JsObject ? (JsObject) obj : null;
  }

  interface Static {
    String[] keys(JsObject obj);
    JsObject defineProperties(JsObject obj, JsObject props);
  }

  interface constructors {
    void constructor(String param);
  }

  boolean hasOwnProperty(String prop);
  boolean isPrototypeOf(JsObject prop);
}

​

then with the annotation processor following code will be generated:

interface Auto_JsObject {

    public static JsObject newInstance(String param) {
      return js("new Object($0)", param);
    }

    public static String[] keys(JsObject obj) {
      return js("Object.keys($0)", obj);
    }

    public static JsObject defineProperties(JsObject obj, JsObject props) {
      ...
    }

    @PrototypeOfJsType(JsObject.class)
    public static class prototype {
       JsObject constructor(String param) { return null}
       boolean hasOwnProperty(String prop) { return false; };
       boolean isPrototypeOf(JsObject prop) { return false; };
    }
}

​

And it looks like following when used:

  MyObject extends JsObject.prototype {}

  JsObject.keys( ... );

  JsObject.newInstance( ... );

  JsObject.of(new Object());

​

One of the advantages of this is; I can additionally generate a special method named statics() that returns an instance of JsObject.Static where we forward all calls to the static methods. This is pretty useful for people who care about pure unit testing.

Although this solution is not ideal due to need for extending a separate class (i.e. JsObject.prototype), it looks reasonably good at first sight.

But there is one major gotcha: it doesn't work!

The reason is, according to here unlike static methods in real classes, static methods in interfaces are not 'inherited' by subclasses.

So I stepped backed and started questioning what we are doing here.

We basically creating a DSL but this DSL is trying to enhance the very same class where it is used. This model isn't directly supported with annotation processors (that's why we do the trick of extending generated class) and we already decided not to use Lombok hacks so it looks like we can't do much here.

So I'm thinking; as we are already defining a DSL why not make it more explicit in a separate class and let it generate the actual API?

Here we go:

@JsApi("Object") // A new annotation for the DSL, not required to be in the SDK
interface __JsObject { // Used underscore so it doesn't show up in code completion

  String[] keys(JsObject obj);

  JsObject defineProperties(JsObject obj, JsObject props);

  static JsObject of(Object obj) {
    return obj instanceof JsObject ? (JsObject) obj : null;
  }

  interface prototype {
    boolean hasOwnProperty(String prop);
    boolean isPrototypeOf(JsObject prop);
  }

  interface constructors {
    void constructor(String param);
  }
}

​

generates following:

@JsType(prototype = "Object")
interface JsObject {
  static JsObject newInstance(String param) {
    return js("new Object($0)", param);
  }

  static String[] keys(JsObject obj) {
    return js("Object.keys($0)", obj);
  };

  static JsObject defineProperties(JsObject obj, JsObject props) {
     ...
  }

  static JsObject of(Object obj) {
    return __JsObject.of(obj);
  }

  boolean hasOwnProperty(String prop);
  boolean isPrototypeOf(JsObject prop);

  public static JsObject defineProperties(JsObject obj, JsObject props) {
    ...
  }

  @PrototypeOfJsType(JsObject.class)
  public static class prototype {
     JsObject constructor(String param) { return null;}
     boolean hasOwnProperty(String prop) { return false; };
     boolean isPrototypeOf(JsObject prop) { return false; };
  }

}

​

So this is basically a working version of the previous one.

One may argue that if we are committing to use some kind of DSL why not use something else like IDL or xml etc. There are 3 main reasons to use java + APT instead of others:

First, our user base already familiar with java so they don't need to learn anything new other than a few simple naming conventions. Also developers need to able to write java code snippets for helper functions (see JsObject.of) which plays well with this approach.

Second, IDE will give free syntax highlighting, checks and auto completion for whole process.

Third, logistics for APT based code generation is already provided by IDEs, build systems and very likely will be provided by superdev in the future.

Long story short; this is quite feasible and viable option but there are still some stuff I don't like about it:

 - JsObject.protoype is still an annoying artifact that all API consumers need to know about.

 - I don't like the compiler silently ignoring some method bodies (i.e. JsObject.prototype::*).

 - There is plenty of code generation going on which causes extra indirections.

So I stepped back one more time and decided give a short break to my strong prejudice on native methods and created a solution around native methods: 

@JsType(prototype="Object")
public class JsObject {

  @JsConstructor
  public static native JsObject newInstance(String param);

  public static native String[] keys(JsObject obj);

  public static native JsObject defineProperties(JsObject obj, JsObject props);

  public static JsObject of(Object obj) {
    return obj instanceof JsObject ? (JsObject) obj : null;
  }

  public native boolean hasOwnProperty(String prop);
  public native boolean isPrototypeOf(JsObject prop);
}

​

This solution is perfect from a purist point of view. No code generation involved; we basically utilize the host language's extension point created solely for this purpose. Here we are saying that this methods exist in the runtime and implemented natively by the platform.

Pros:

It is intuitive for java developers. There is no magic prototype class to extend from, no conventions or DSL, no code generators. It is more explicit and it is very clear where the native boundary is when you look at the code. Probably it is even more pleasing to implement in the compiler.

All the cons are around the testability aspect. For JRE testing, native methods are a headache. Also we no longer generate an interface for static methods. We are basically going back to square one with respect to testability.

But on the other hand it is not horrible. For people who insists on JRE testing; we can provide a classloader that stubs all native methods or they can use a more powerful mocking library like powermock or gwt-mockito.

I would like to first get high level feedback here before discussing this internally. I'll update the design doc if we conclude on something.

- Goktug

[Manuel Carrasco Moñino]

After reading, and playing with some stub code based on these ideas, I come to the conclusion than 'native' is almost perfect from the point of view of a gwt developer.

We want pure java syntax and simplicity, and any other conventions like DSLs or magic interface names, etc, could obscure the understanding existing code,  or trying to figure out how to implement something.

For java developers 'native' means that the platform knows how to deal with this method, and in the case of gwt this is true and intuitive, meaning that 'this is native because it's implemented natively in JS'.

The compiler ability to automatically resolve native methods, could be very useful not only for JsInterop but for many other cases where we use them.

Hence any getter or setter could be automatically written by the compiler if it does not have the jsni fragment. For instance in c.g.g.d.c.StyleElement we could get rid of all of the jsni fragments and just write methods declaration

 public final native void setCssText(String cssText); 

If the compiler does this, we could save thousands of line of code in jsni fragments in the gwt core, also in elemental, and in several other 3 party libraries.

Even, we could have an annotation to deal with simple JSNI code instead of using c++ comment blocks to change the default implementation:

 @Jsni("return $doc.head || $doc.getElementsByTagName('head')[0]")

 public final native void getHead();

Also it would be nice if the compiler is able to deal directly with type conversion and it is able to box returned values: java arrays to jsArrays, js numbers to java types, etc. This will make gwt developer experience better.

Related with testing, I think that because we already use the GWTTestCase to run JRE tests (returning module null), it should be not difficult to modify it and return proxies of classes implementing native methods ant throwing sensible messages to the user telling what methods are used, etc.

Coming back to JsInterop, I agree that native methods is an easy solution, probably not so difficult to implement, and definitively solves many problems like inheritance, static methods, constructors, etc.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/CAN%3DyUA2DvnfqtL1F4zd_ydM4KdWx9bBXCdtY7RGs_vaVvh88%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Stephen Haberman]

> I will try to summarize my thought process and different options that
> I have played with so that you could get a better understanding where
> I'm coming from and I hope it will provide good documentation for
> future.

Thanks for the email; I think the format was really useful.

> One may argue that if we are committing to use some kind of DSL why
> not use something else like IDL or xml etc. There are 3 main reasons
> to use java + APT instead of others:

I really want to advocate APT, as I've used it and do generally like it,
but frankly it can be a huge PITA for Eclipse. See long-standing issues
e.g.:

https://github.com/square/dagger/issues/126

Granted, maybe someone can just fix Eclipse, but, in my experience, it
can really ruin the first impressions for a project (1000s of compile
errors due to missing generated code, and no hints that, btw, it's
because the APT was not on the classpath/didn't run).

> All the cons are around the testability aspect. For JRE testing,
> native methods are a headache. Also we no longer generate an interface
> for static methods.

I know I just bitched about APT, but, just musing, what if the class
(with the native methods/etc.) was the DSL the user wrote, and then APT
just generated an interface from that?

This is basically what I'm doing by hand in tessell with writing
IsTextBox for TextBox, etc.

I admittedly am not an expert on JS prototype semantics, so am not sure
how this would handle statics/etc. But in testing I usually only care
about instance methods anyway...

A few years ago I spiked an APT processor to do this automatically:

https://github.com/stephenh/interfacegen

The oddity then becomes that the user is writing "class JsObject", but
then if they want testable code, they need to code against the generated
"IsJsObject" interface. But perhaps that it's optional is a good thing,
e.g. you could come back and add the "@GenInterface"-type method later
if/when you wanted.

Anyway, I think I do like the last proposal, the class/native method
best.

Thanks, Goktug!

- Stephen

[Goktug Gokdogan]

There is also a third option.

User writes:

@JsType(prototype = "Object")
interface JsObject {

  class prototype extends Prototype_JsObject {}

  

interface Static {
    String[] keys(JsObject obj);
    JsObject defineProperties(JsObject obj, JsObject props)

;
  }

  static Static getStatic() {
    return new Static_JsObject();
  }

  

static JsObject of(Object obj) {
    return obj instanceof JsObject ? (JsObject) obj : null

;
  }

  @JsConstructor

  void constructor(String param);

  boolean hasOwnProperty(String prop);
  boolean isPrototypeOf(JsObject prop);
}

​

which generates:

@PrototypeOfJsType(JsObject.class)
public class Prototype_JsObject {
   

JsObject constructor(String param) { return null;}
   boolean hasOwnProperty(String prop) { return false; }
   boolean isPrototypeOf(JsObject prop) { return false; }
}

public class Static_JsObject implements Static {
  

JsObject newInstance(String param) {
    return js("new Object($0)", param)

;
  }

  public String[] keys(JsObject obj) {
    

return js("Object.keys($0)", obj)

;
  };

  public JsObject defineProperties(JsObject obj, JsObject props) {
    ...
  }
}

​

And usage looks like:

  MyObject extends JsObject.prototype {}

  JsObject.getStatic().keys( ... );

  JsObject.getStatic().newInstance( ... );

  JsObject.of(new Object());

​

And it is perfectly testable.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/20141101102535.45715a48%40sh9.

[Goktug Gokdogan]

(I will read & respond to comments after the weekend)

[Ray Cromwell]

Moving from interfaces to classes has other downsides. For example, you can't avoid not generating a Java class/object, you can't have your own super-class hierarchy and mark your class as supporting the same interface as an external JS implemented one, the notion of JS objects "implementing" a Java interface would seem strange if they implement the interface without subclassing the prototype of the Java object with the annotation.

Take something like Elemental 2.0, which is likely to have several hundred, maybe 1000 types for every conceivable WebUI, we do not want defineClass(), class literals, castmaps, or java.lang.Object stuff set up for any of them. With interfaces, it's clear there's no backing interface to compile. With classes, we'd have to add some hackery to the compiler like "if it's a jstype, and it has zero implemented methods, and it's all native, and it doesn't extend another class that has non-native methods, *then* we can just drop it's class setup from the output".

Otherwise, for every single one of these you'll get massive bloat, and a SDM compile of Elemental 2.0 would have a thousand stubbed out classes in it.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/CAN%3DyUA30t6RmeVs%3D9XGOp4HU0_u_NYu4J42SYJ9HQnH9PgoXSg%40mail.gmail.com.

[Goktug Gokdogan]

On Sat, Nov 1, 2014 at 3:33 AM, Manuel Carrasco Moñino <man...@apache.org> wrote:

After reading, and playing with some stub code based on these ideas, I come to the conclusion than 'native' is almost perfect from the point of view of a gwt developer.

We want pure java syntax and simplicity, and any other conventions like DSLs or magic interface names, etc, could obscure the understanding existing code,  or trying to figure out how to implement something.

For java developers 'native' means that the platform knows how to deal with this method, and in the case of gwt this is true and intuitive, meaning that 'this is native because it's implemented natively in JS'.

The compiler ability to automatically resolve native methods, could be very useful not only for JsInterop but for many other cases where we use them.

Hence any getter or setter could be automatically written by the compiler if it does not have the jsni fragment. For instance in c.g.g.d.c.StyleElement we could get rid of all of the jsni fragments and just write methods declaration

 public final native void setCssText(String cssText); 

If the compiler does this, we could save thousands of line of code in jsni fragments in the gwt core, also in elemental, and in several other 3 party libraries.

Even, we could have an annotation to deal with simple JSNI code instead of using c++ comment blocks to change the default implementation:

 @Jsni("return $doc.head || $doc.getElementsByTagName('head')[0]")

 public final native void getHead();

I don't have any intention to go beyond what java supports in terms of native methods. The JSNI annotation just add another obscure way of doing what could do with js(....) statements.

 

Also it would be nice if the compiler is able to deal directly with type conversion and it is able to box returned values: java arrays to jsArrays, js numbers to java types, etc. This will make gwt developer experience better.

This is in the plans and we are planning to support something like JsConvert (see the doc).

 

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/CAM28XAtxB%3DZwRGfKxGVN-qa-%3DKk1jSZAMsq-0zsKTGRWuqzFZw%40mail.gmail.com.

[Goktug Gokdogan]

On Sat, Nov 1, 2014 at 8:25 AM, Stephen Haberman <stephen....@gmail.com> wrote:

> I will try to summarize my thought process and different options that
> I have played with so that you could get a better understanding where
> I'm coming from and I hope it will provide good documentation for
> future.

Thanks for the email; I think the format was really useful.

> One may argue that if we are committing to use some kind of DSL why
> not use something else like IDL or xml etc. There are 3 main reasons
> to use java + APT instead of others:

I really want to advocate APT, as I've used it and do generally like it,
but frankly it can be a huge PITA for Eclipse. See long-standing issues
e.g.:

https://github.com/square/dagger/issues/126

Granted, maybe someone can just fix Eclipse, but, in my experience, it
can really ruin the first impressions for a project (1000s of compile
errors due to missing generated code, and no hints that, btw, it's
because the APT was not on the classpath/didn't run).

GWT already requires kind of special setup in Eclipse so I think we are in better shape.

Also we will need APT's regardless of this feature as that will be the only way for code generation in the long run.

 

> All the cons are around the testability aspect. For JRE testing,
> native methods are a headache. Also we no longer generate an interface
> for static methods.

I know I just bitched about APT, but, just musing, what if the class
(with the native methods/etc.) was the DSL the user wrote, and then APT
just generated an interface from that?

This is basically what I'm doing by hand in tessell with writing
IsTextBox for TextBox, etc.

I admittedly am not an expert on JS prototype semantics, so am not sure
how this would handle statics/etc. But in testing I usually only care
about instance methods anyway...

A few years ago I spiked an APT processor to do this automatically:

https://github.com/stephenh/interfacegen

The oddity then becomes that the user is writing "class JsObject", but
then if they want testable code, they need to code against the generated
"IsJsObject" interface. But perhaps that it's optional is a good thing,
e.g. you could come back and add the "@GenInterface"-type method later
if/when you wanted.

Anyway, I think I do like the last proposal, the class/native method
best.

Thanks, Goktug!

- Stephen

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/20141101102535.45715a48%40sh9.

[Goktug Gokdogan]

I think I didn't understand some of your concerns. Let's meet some time this week to flush out what all of this means and the implications.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/CAPVRV7eyW-Xw8SHefBOHff6ttUgh5X20ghSn0A9t9V_vcv0aHw%40mail.gmail.com.

[Goktug Gokdogan]

An additional pros / cons about getting rid of JsObject.prototype:

Not having JsObject.prototype fixes the symmetry problem between SDK and user code. For example earlier, when you extend Element you need to extend Element.prototype but if the custom element is implemented in java side, then you need to extend MyJavaElement without the .prototype.

The counter argument could be; extending .prototype makes it obvious that your are extending a javascript object (which has its own pros and cons (breaks encapsulation vs one might want to know this for some reason that I couldn't find yet)).

 

[Cristian Rinaldi]

@goktug I like this option, it is expressive and concise.

But I have a question, 

The code generated for the prototype, suppose we are extending some existing JS functionality and want to add some function, the prototype generated will have native code?, how this prototype generates subyasente invoking the JS function?

by example:

@JsType(prototype = "Document")
public interface Document {

    public HTMLElement createElement(String div);

    public HTMLElement getElementsByTagName(String body);

    @JsProperty
    public HTMLBodyElement getBody();

    public NodeList querySelector(String selector);

    public NodeList querySelectorAll(String selector);

    class prototype extends Prototype_Document {}
   
    ....
    ....
}

@PrototypeOfJsType(Document.class)
public class Prototype_Document{
    ....
    ....
    public NodeList querySelectorAll(String selector){
       //The question (How will called underlying code?)    
    }
}

public class MyDocument extends Document.prototype{
 
          public void myMethod(){
               //TODO
          }

}

Cheers :)

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.

[Goktug Gokdogan]

It is compiler magic. The compiler understands from PrototypeOfJsType and assumes any methods in the class are native, hence ignores the method body.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/a10b0e54-eea9-4886-bfc0-1103eede4be2%40googlegroups.com.

[Cristian Rinaldi]

@goktug

Thanks for the answer!!

[Julien Dramaix]

Hi all,

Finally, did you made a decision ? 

I like very much the native methods approach for its simplicity and a test library like PowerMock seems to be able to mock native methods. But Ray seems to have some concerns about using class instead of Interface. So what was the selected approach ?

Julien

[Goktug Gokdogan]

I think last time I checked everybody was convinced this is the way to go and we are planning towards this. 

I'll update the spec when I have the chance.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/2ccff98a-eb73-40b3-8535-ef456eec5ac7%40googlegroups.com.