Is there any way to provide missing implementations?

141 views
Skip to first unread message

Nacho Cordón

unread,
Mar 15, 2024, 5:38:39 AM3/15/24
to TeaVM
Hello!

When we transpile some code with TeaVM we can see it's missing some implementations:

  • java.util.WeakHashMap: I can see a pr trying to add it was never accepted
  • java.util.concurrent.CopyOnWriteArrayList
  • java.io.InvalidClassException, java.util.CancellationException
  • PrintStream.format, PrintStream.printf, String.format
Some of those I can maybe work on an implementation to add to TeaVM as a contribution if the maintainers are open to the idea? 

Is there any way to provide our own implementations when they are missing in our code? 

Best,
Nacho

Alexey Andreev

unread,
Mar 15, 2024, 1:11:14 PM3/15/24
to te...@googlegroups.com

Hi


When we transpile some code with TeaVM we can see it's missing some implementations:

By saying "we" whom do you mean? Can you introduce yourself?


  • java.util.WeakHashMap: I can see a pr trying to add it was never accepted
Sure. This PR is totally broken and I had no feedback from the author. Perhaps due to misunderstanding. I honestly tried to figure out what he was proposing, but apparently he just decided I don't want to review his PR. Anyway, WeakHashMap is in 0.10.0 milestone, and it's one of two still not resolved issues for the milestone. So I think it's highly likely that it's been resolved somewhere withing a month.
  • java.util.concurrent.CopyOnWriteArrayList
This one is really tough. One needs to understand how TeaVM virtual thread work under the hood and how to implement thread emulation code that would avoid async infection in single-thread scenario. Also, it's usually quite challenging to write tests properly.
  • java.io.InvalidClassException, java.util.CancellationException

These are trivial

  • PrintStream.format, PrintStream.printf, String.format

I believe these could be implemented by simple delegation to the functionality that is already implemented. I'm only not sure which one of overloads you need, for example, for String.format.

Some of those I can maybe work on an implementation to add to TeaVM as a contribution if the maintainers are open to the idea?

You can work on implementation, but be prepared that it might be much more difficult than you imagine. According to my experience only very few developers managed to push their changes to the review process.


Is there any way to provide our own implementations when they are missing in our code?

Yes. If it's a whole class, then simply take a look at `classlib` module and follow same naming pattern. Simply put missing class in your own codebase and this should work. For individual method of a class there's no solution.

And having own implementation of a class can make things a bit easier: for example, you not necessarily need general-purpose implementation of `CopyOnWriteArrayList` with all the corner cases properly covered. In this case you may write a (wrong) implementation that simply satisfies you. Of course, in the latter case I won't take such code in TeaVM code base.


Another way for you it to motivate me to implement these classes. For example, you can convince me that helping you can make a lot of buzz around TeaVM. Or you could become a sponsor.

Nacho Cordón

unread,
Mar 17, 2024, 8:00:45 AM3/17/24
to TeaVM

▶️ By saying "we" whom do you mean? Can you introduce yourself?

I'm a software engineer at Neo4j. We are working on a language server in Typescript, and we are transpiling (it's all experimental at the moment) part of the parsing stack of the database (JVM based) to provide for semantic analysis of queries. It was working fine with an old parser technology (which I suppose it was using very vanilla like Java), the new version (based on antlr4) is breaking because it's missing some of the implementations I listed.

▶️ Yes. If it's a whole class, then simply take a look at `classlib` module and follow same naming pattern. Simply put missing class in your own codebase and this should work. For individual method of a class there's no solution.

I think I found a way to circumvent that, even for missing methods. Creating a project (in our case maven project) that as you said overrides the classes we are missing but also just duplicates the code of the ones that are missing the methods (adding those missing methods) and importing that project instead of teavm-classlib seems to at least making it compile. But of course I'm cutting corners because we don't need a full functioning `CopyOnWriteArrayList` implementation for example.

▶️ Another way for you it to motivate me to implement these classes. For example, you can convince me that helping you can make a lot of buzz around TeaVM. Or you could become a sponsor

I had been chasing setting up a small sponsorship and I got that approved on Friday, but I understand it's not a big allocation so I would not expect that to be enough of a motivation. It's just to show we are really thankful for your project. Hopefully I can contribute some of those to the codebase myself. And if our project was successful even advertise how awesome TeaVM is!!

Best,
Nacho 

Alexey Andreev

unread,
Mar 19, 2024, 3:48:02 AM3/19/24
to te...@googlegroups.com

▶️ By saying "we" whom do you mean? Can you introduce yourself?

I'm a software engineer at Neo4j. We are working on a language server in Typescript, and we are transpiling (it's all experimental at the moment) part of the parsing stack of the database (JVM based) to provide for semantic analysis of queries. It was working fine with an old parser technology (which I suppose it was using very vanilla like Java), the new version (based on antlr4) is breaking because it's missing some of the implementations I listed.

BTW, this project uses its own fork of antlr to properly compile with TeaVM.

▶️ Yes. If it's a whole class, then simply take a look at `classlib` module and follow same naming pattern. Simply put missing class in your own codebase and this should work. For individual method of a class there's no solution.

I think I found a way to circumvent that, even for missing methods. Creating a project (in our case maven project) that as you said overrides the classes we are missing but also just duplicates the code of the ones that are missing the methods (adding those missing methods) and importing that project instead of teavm-classlib seems to at least making it compile. But of course I'm cutting corners because we don't need a full functioning `CopyOnWriteArrayList` implementation for example.

I just thought about CopyOnWriteArrayList, and it turns out that during write operations no context switches possible (like in ConcurrentHashMap, for instance, which can call `hashCode`, which in turn can pause execution). So basically CopyOnWriteArrayList is more or less ArrayList in TeaVM, with couple additional methods and restricted ListIterator functionality. I'll try to implement this.

▶️ Another way for you it to motivate me to implement these classes. For example, you can convince me that helping you can make a lot of buzz around TeaVM. Or you could become a sponsor

I had been chasing setting up a small sponsorship and I got that approved on Friday, but I understand it's not a big allocation so I would not expect that to be enough of a motivation. It's just to show we are really thankful for your project. Hopefully I can contribute some of those to the codebase myself. And if our project was successful even advertise how awesome TeaVM is!!

I saw this, and now I have explanation why neo4j became a sponsor. Thanks! Sure, $25 is not enough to cover my work on these classes. However, this and the fact that TeaVM gets some more publicity is a good enough reason.

Alexey Andreev

unread,
Mar 21, 2024, 2:21:29 PM3/21/24
to TeaVM
As for `String.format`, it's supported in TeaVM. Did you mean `String.formatted`?

вторник, 19 марта 2024 г. в 08:48:02 UTC+1, Alexey Andreev:

Alexey Andreev

unread,
Apr 9, 2024, 3:04:21 PM4/9/24
to TeaVM
I just implemented all of the classes/methods you listed here, except for `String.format`, which was actually there. Can you please try with TeaVM build from master branch? I'll create the next preview build a bit later.

четверг, 21 марта 2024 г. в 19:21:29 UTC+1, Alexey Andreev:

Nacho Cordón

unread,
Apr 10, 2024, 5:39:33 PM4/10/24
to TeaVM
Thanks a lot for that Alexey! I'll give a try to transpiling using the master branch this / next week.

Best,
Nacho

Nacho Cordón

unread,
Apr 24, 2024, 11:22:02 AM4/24/24
to TeaVM
Amazing! The new added implementations worked perfectly and allowed us to transpile a whole parser made in antlr4, which is pretty impressive.

Best,
Nacho

Reply all
Reply to author
Forward
0 new messages