Need advice to create an executable with Scalismo/scala

[Maia R.]

Dear community,

I'd like to create an executable of my Scalismo-based app so that I can use it outside of Scalismo (in fact in a C++-based application). I saw that this could be done through java but I am not sure.

Had someone already worked on this before ? How can this be done ?

Also, is it better to use an object with main function for such an executable or an object which extended App trait is good as well ?

Thank you,

Best regards

Maia

[Marcel Luethi]

Dear Maia

I don't have much experience creating native images, but I did a quick test using graalvm and succeeded in producing a native image.

The way it worked for me was to build a jar file using sbt-assembly and then to use graal's native-image command:

https://www.graalvm.org/reference-manual/native-image/ 

I have only tested it on linux, but chances are that it also works on windows.

Best regards,

Marcel

--
You received this message because you are subscribed to the Google Groups "scalismo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalismo+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/b857cc4e-7354-4ae3-8b2a-2a015376ce0en%40googlegroups.com.

[Maia R.]

Thank you very much Marcel,

I will check that.

Best regards

Maia

[Maia R.]

Hi Marcel,

May I ask you how you did that with a simple example and the steps you used ?

It's still unclear for me how to use GraalVM. For example, if I have a c++ project with solution to be build with Visual Studio, how can I add/use the jar file from sbt-assembly within that project ? Do I need to have a main in the Scala files or just a bunch of Scala classes ?

Thank you very much,

Best regards

Maia

Le samedi 10 juillet 2021 à 10:28:29 UTC-4, marcel...@gmail.com a écrit :

[Marcel Luethi]

Dear Maia

Unfortunately I did not get that far with my experiments. I only tried out if it is at all possible to compile scalismo to native code and that seems to be the case. But I have not investigated how one would call it from C++. If you want to investigate that route further, I would try first with a hello world project, where you simply compile one scala file to native and call that from c++. If this works, you can try if Scalismo can be called in a second step.

I (and most likely many other members of this mailing list) would be very curious to here what comes out of your experiments.

One more thing: If your problem is not about calling scalismo directly from C++ but just using the functionality you implemented in Scalismo, then things are fairly simple. You can use thrift (https://thrift.apache.org/) to create an interface for your service and then run it as a separate process on a normal jvm. You could then use this service from many different languages, including C++ and Python. We have used this approach previously and our experience was very positive. Compared to the native solution, it comes with two benefits: 1) you can also run it over a network 2) as the server runs as a separate process, it keeps running even if your c++ application segfaults :-)

Best wishes,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/495bf0f1-f960-4e36-8cf8-4a4df4a45babn%40googlegroups.com.

[V R]

Dear Marcel,

Thank you  very much. I will certainly give feedback here.

Best regards

Maia

[Maia R.]

Dear Marcel,

May I ask you why did you choose Thrift over gRPC for example ?

Someone one Scala users suggests gRPC.

Thank you,

Best regards

[Marcel Luethi]

Dear Maia

gRPC is another option that does pretty much the same as thrift. At the point where we made the choice, we choose Thrift because it made it possible to serialize large objects (eg. 3D images) directly, whereas in protobuf the payload of a message is limited. However, as far as I know, you can send large objects as streams using gRPC, so it is not really a limitation of gRPC. As there is excellent support for gRPC in Scala I think it seems like a good option too. But I don't have any experience with it.

Best,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/2e519c05-8fa7-488a-90c1-f86e10d07d30n%40googlegroups.com.

[Maia R.]

Thank you Marcel.

Best regards

[Maia R.]

Dear Marcel,

I'd like to ask a general question about your implementation here https://github.com/marcelluethi/ui-service/tree/master/src/main/scala.

Suppose for example that I have a Scalismo function to reconstruct a mesh, so I need a ssm, the partial mesh etc, all written with Scalismo API:

def reconstruct(

ssm: PointDistributionModel[_3D],
initialInstance: TriangleMesh[_3D],
partialMesh : TriangleMesh[_3D],
nbOfSamplePoints: Int,
params: Seq[ReconstructParameters],
ssmView: PointDistributionModelViewControlsTriangleMesh3D) : TriangleMesh[_3D] = ???

This function returns the reconstructed mesh that I want to use in a C++ external project.

Is it mandatory to create/rewrite this whole reconstruction function with Thrift interface as defined in .thrift file such as  in Serialization.scala and Server.scala ? What is the main use of Serialization.scala ?

I thought that only data as computed at each side are transfered between Client/Server ... 

Sorry if this is an obvious question ...

Thank you,

Best regards. 

[Marcel Luethi]

Dear Maia

You are correct, you only need to serialize the objects you are transferring. In the Scalismo UI example, the idea was to expose the full visualization api to C++. This is why you find so many objects that are serialized.

Best,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/259cca00-de83-4a5d-aa04-998db563d170n%40googlegroups.com.

[V R]

Dear Marcel,

Thank you for the explanation.

Best regards.

[Maia R.]

Dear Marcel,

I better understand how Thrift works and managed to build a server-client all in C++ (Visual Studio) by autogenerating cpp codes with: thrift --gen cpp service.thrift.

After looking at your examples in https://github.com/marcelluethi/statismo-ui and https://github.com/marcelluethi/ui-service, I'd like to have more precisions on how did you get thoses codes.

As far as I understand, Thrift generated cpp/python/java... codes from a service (*.thrift) but as it does not support Scala, Scooge is used instead: is that right ? 

Could you be more specific about your steps please:

Did you defined a single ui.thrift and use it to:

1. create scala codes from scrooge

2. create c++ codes from thrift

Thank you very much,

Best  regards,

Maia

[Marcel Luethi]

Dear Maia

Yes, from the Scala side you generate the code using Scrooge. It's been a while that I did it last time, but I think you just type sbt scroogeGen and then it generates all the files for you.  On the C++ side there is a generator that comes with thrift (see e.g. https://thrift.apache.org/tutorial/cpp.html).

Best regards,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/e14abe20-f309-470f-8bbd-acf3ca0357c0n%40googlegroups.com.

[V R]

Thank you very much.

Best regards

Maia

[Maia R.]

Dear Marcel,

I'd like to know if this file (or a template of it) was obtained after calling >sbt scroogeGen

https://github.com/marcelluethi/ui-service/blob/master/src/main/scala/server/Server.scala

I followed the same process as yours and on Scrooge website with a very simple service:

service HelloSvc {
string hello_func()
}

But I got 2 very "complicated" classes, named HelloSvc$FinagleClient.scala and HelloSvc$FinagleService.scala. 

https://scastie.scala-lang.org/ojqjfUYTTXeleCDZ9zOs1w

https://scastie.scala-lang.org/KG8TgmheRVqT1KOIFq16Ug

I used similar build.sbt/plugins.sbt to yours:

https://scastie.scala-lang.org/UceQsvtJQV21faHnzi8qVQ

https://scastie.scala-lang.org/sB9LGMx5SByfjK3F13q0IQ

I'm wondering if it comes from the default Finagle https://twitter.github.io/scrooge/SBTPlugin.html#plugin-addition

but the generated codes for the service are really different from what I saw elsewhere for other languages (ex: cpp or python for which server skeleton codes (templates) are generated with the port and all Thrift stuffs ...)

Thank you

Best regards

Maia

[Marcel Luethi]

Dear Maia

Unfortunately I am at the moment unable to support you with Thrift.  I am a bit reluctant because this is not regarding Scalismo and shape modeling. Furthermore (and maybe more important) I have not worked on it for a while and it would need to dig into it myself, to be able to answer competently. This is just not possible for me at this time, due to my current teaching load at the university. I can, however, refer you to shapemeans. They can provide you with commercial support regarding thrift, and also give you further advise on what to look for when deploying an application. 

Best,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/e2e14be7-579d-4409-9000-8284b5612fc9n%40googlegroups.com.

[Maia R.]

Dear Marcel,

I understand. Have a good session.

Best regards,

Maia

[Maia R.]

Dear Marcel,

Just one last question before I close this very long exchange (thank you a lot for your patience): why not use JNI (Java Native Interface) instead of Thrift or gRPC ?

Thank you,

Best regards

Maia

[Marcel Luethi]

Hi Maia

JNI is a lot harder to use in my opinion and less flexible.

Thrift  or gRPC nicely separate the processes, enforce a clean interface  and even allow you to run the services on different machines. 

Another advantage of Thrift is, that once you have set up your scalismo app as a thrift service, you can call it from almost any other programming language.

Best regards,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/05fe17fe-8a24-4686-a166-d8a8870cdd6fn%40googlegroups.com.

[V R]

Dear Marcel,

Thank you very much.

Best regards

Maia

[Maia R.]

Dear Marcel and community,

First of all, Happy new year !

Marcel, when you use the word "direcly" when you said "it is possible to serialize large objects (eg. 3D images) directly" , what is the the intent of the word 'directly' here ? 

Best regards,

Maia

[Marcel Luethi]

Dear Maia

Thank you, a happy new year to you too.

What I meant with "directly" is that you do not have to split it up into smaller chunks before sending it. You can just send the whole image at once. Some other technologies cap the payload of an individual message, to, say, 7 MB per message. In thrift there does not seem to be such a limit.

Best regards,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/0f3a7e91-18bf-4997-9a4c-d65a319737a8n%40googlegroups.com.

[Maia R.]

Dear Marcel,

Sorry I didn't find your message earlier.

I had a lot of run time errors with a simple example of server-client with Finagle-Thrift-Scrooge. 

I found that there is no active support at all with this API compared to gPRC so we decided to go with gRPC, documentation are also outdated, versions are not following, the community is not active at all, etc... 

I found your example of ui-service very valuable though, so thank you very much.

Best regards

Maia

[V R]

Dear Marcel,

In fact, it was working when I created a client-server all in Scala, or all in C++. The issues appear (crash on run time) when the server is in Scala and the client in C++ and vice-versa.

At the moment, people working with gRPC said that messages up to several MB are okay to be transfered with gRPC.

Again, thank you.

Best regards,

Maia

[Marcel Luethi]

Dear Maia

Thanks for reporting back. It is great to hear that gRPC works.Although I believe that Thrift should be a stable and production ready solution, it is always great to have a more modern alternative that has better documentation. And in a recent grpc benchmark (https://www.nexthink.com/blog/comparing-grpc-performance/) scala even turns out to lead the pack in terms of performance.

I am sure the interoperability with C++ is a topic that many people are interested in and whatever experience you can share on this list is greatly appreciated.

Best regards,

Marcel

[V R]

Dear Marcel,

Thank you for your support.

I will certainly share whatever I can. 

First thing I noticed is about IDL file, *.thrift is better than *.proto file. So, more work is to be done. I didn't finished yet ...

Best regards,

Maia

[V R]

Dear Marcel 

Thank you for sharing this recent gRPC performance. It's really interesting.

Best regards,

Maia