Huge native images?

[Qbyte Consulting]

Hi,

Our native image runner is nearly 55MB and takes our build server over 7 minutes to compile (on a MacBook Pro less than 2 minutes). Our app creates only a tiny 60kb jar.

Are there ways to optimize compilations? For example by using precompiled code or dependencies, incremental builds, caching previously compiled modules, excluding unused dependencies?

In appname-native-image-source-jar/lib there are even cucumber jars*? Surely not required at app runtime?

regards,

John

* dependencies

ch.obermuhlner.big-math-2.0.1.jar

com.fasterxml.jackson.core.jackson-annotations-2.11.3.jar

com.fasterxml.jackson.core.jackson-core-2.11.3.jar

com.fasterxml.jackson.core.jackson-databind-2.11.3.jar

com.fasterxml.jackson.dataformat.jackson-dataformat-yaml-2.11.3.jar

com.fasterxml.jackson.datatype.jackson-datatype-jdk8-2.11.3.jar

com.fasterxml.jackson.datatype.jackson-datatype-jsr310-2.11.3.jar

com.fasterxml.jackson.jaxrs.jackson-jaxrs-base-2.11.3.jar

com.fasterxml.jackson.jaxrs.jackson-jaxrs-json-provider-2.11.3.jar

com.fasterxml.jackson.module.jackson-module-jaxb-annotations-2.11.3.jar

com.fasterxml.jackson.module.jackson-module-parameter-names-2.11.3.jar

com.github.java-json-tools.btf-1.3.jar

com.github.java-json-tools.jackson-coreutils-2.0.jar

com.github.java-json-tools.json-patch-1.13.jar

com.github.java-json-tools.msg-simple-1.2.jar

com.github.javaparser.javaparser-core-3.13.10.jar

com.github.virtuald.curvesapi-1.06.jar

com.google.protobuf.protobuf-java-3.13.0.jar

com.ibm.async.asyncutil-0.1.0.jar

com.jayway.jsonpath.json-path-2.4.0.jar

com.sun.activation.jakarta.activation-1.2.1.jar

com.sun.istack.istack-commons-runtime-3.0.10.jar

com.thoughtworks.xstream.xstream-1.4.11.1.jar

com.zaxxer.SparseBitSet-1.2.jar

commons-codec.commons-codec-1.14.jar

commons-io.commons-io-2.8.0.jar

io.cucumber.create-meta-2.0.4.jar

io.cucumber.cucumber-core-6.9.1.jar

io.cucumber.cucumber-expressions-10.3.0.jar

io.cucumber.cucumber-gherkin-6.9.1.jar

io.cucumber.cucumber-gherkin-messages-6.9.1.jar

io.cucumber.cucumber-plugin-6.9.1.jar

io.cucumber.datatable-3.5.0.jar

io.cucumber.docstring-6.9.1.jar

io.cucumber.html-formatter-9.0.0.jar

io.cucumber.messages-13.2.1.jar

io.cucumber.tag-expressions-3.0.0.jar

io.netty.netty-buffer-4.1.49.Final.jar

io.netty.netty-codec-4.1.49.Final.jar

io.netty.netty-codec-dns-4.1.49.Final.jar

io.netty.netty-codec-http-4.1.49.Final.jar

io.netty.netty-codec-http2-4.1.49.Final.jar

io.netty.netty-codec-socks-4.1.49.Final.jar

io.netty.netty-common-4.1.49.Final.jar

io.netty.netty-handler-4.1.49.Final.jar

io.netty.netty-handler-proxy-4.1.49.Final.jar

io.netty.netty-resolver-4.1.49.Final.jar

io.netty.netty-resolver-dns-4.1.49.Final.jar

io.netty.netty-transport-4.1.49.Final.jar

io.quarkus.arc.arc-1.10.0.Final.jar

io.quarkus.arc.arc-processor-1.10.0.Final.jar

io.quarkus.gizmo.gizmo-1.0.6.Final.jar

io.quarkus.http.quarkus-http-core-3.0.16.Final.jar

io.quarkus.http.quarkus-http-http-core-3.0.16.Final.jar

io.quarkus.http.quarkus-http-servlet-3.0.16.Final.jar

io.quarkus.http.quarkus-http-vertx-backend-3.0.16.Final.jar

io.quarkus.quarkus-arc-1.10.5.Final.jar

io.quarkus.quarkus-bootstrap-runner-1.10.0.Final.jar

io.quarkus.quarkus-core-1.10.0.Final.jar

io.quarkus.quarkus-development-mode-spi-1.10.0.Final.jar

io.quarkus.quarkus-jackson-1.10.0.Final.jar

io.quarkus.quarkus-netty-1.10.0.Final.jar

io.quarkus.quarkus-resteasy-1.10.5.Final.jar

io.quarkus.quarkus-resteasy-common-1.10.0.Final.jar

io.quarkus.quarkus-resteasy-jackson-1.10.5.Final.jar

io.quarkus.quarkus-resteasy-server-common-1.10.0.Final.jar

io.quarkus.quarkus-security-runtime-spi-1.10.0.Final.jar

io.quarkus.quarkus-smallrye-openapi-1.10.5.Final.jar

io.quarkus.quarkus-swagger-ui-1.10.0.Final.jar

io.quarkus.quarkus-undertow-1.10.0.Final.jar

io.quarkus.quarkus-vertx-core-1.10.0.Final.jar

io.quarkus.quarkus-vertx-http-1.10.0.Final.jar

io.quarkus.security.quarkus-security-1.1.3.Final.jar

io.smallrye.common.smallrye-common-annotation-1.4.1.jar

io.smallrye.common.smallrye-common-classloader-1.4.1.jar

io.smallrye.common.smallrye-common-constraint-1.4.1.jar

io.smallrye.common.smallrye-common-expression-1.4.1.jar

io.smallrye.common.smallrye-common-function-1.4.1.jar

io.smallrye.common.smallrye-common-io-1.4.1.jar

io.smallrye.config.smallrye-config-1.9.3.jar

io.smallrye.config.smallrye-config-common-1.9.3.jar

io.smallrye.reactive.mutiny-0.10.1.jar

io.smallrye.smallrye-open-api-core-2.0.15.jar

io.vertx.vertx-auth-common-3.9.4.jar

io.vertx.vertx-bridge-common-3.9.4.jar

io.vertx.vertx-core-3.9.4.jar

io.vertx.vertx-web-3.9.4.jar

io.vertx.vertx-web-common-3.9.4.jar

jakarta.activation.jakarta.activation-api-1.2.1.jar

jakarta.annotation.jakarta.annotation-api-1.3.5.jar

jakarta.el.jakarta.el-api-3.0.3.jar

jakarta.enterprise.jakarta.enterprise.cdi-api-2.0.2.jar

jakarta.inject.jakarta.inject-api-1.0.jar

jakarta.interceptor.jakarta.interceptor-api-1.2.5.jar

jakarta.servlet.jakarta.servlet-api-4.0.3.jar

jakarta.transaction.jakarta.transaction-api-1.3.3.jar

jakarta.validation.jakarta.validation-api-2.0.2.jar

lib.txt

net.minidev.accessors-smart-1.2.jar

net.minidev.json-smart-2.3.jar

org.antlr.antlr-runtime-3.5.2.jar

org.antlr.antlr4-runtime-4.8.jar

org.apache.commons.commons-collections4-4.4.jar

org.apache.commons.commons-compress-1.20.jar

org.apache.commons.commons-math3-3.6.1.jar

org.apache.poi.poi-4.1.2.jar

org.apache.poi.poi-ooxml-4.1.2.jar

org.apache.poi.poi-ooxml-schemas-4.1.2.jar

org.apache.xmlbeans.xmlbeans-3.1.0.jar

org.apiguardian.apiguardian-api-1.1.0.jar

org.drools.drools-canonical-model-7.46.0.Final.jar

org.drools.drools-compiler-7.46.0.Final.jar

org.drools.drools-core-7.46.0.Final.jar

org.drools.drools-core-reflective-7.46.0.Final.jar

org.drools.drools-decisiontables-7.46.0.Final.jar

org.drools.drools-model-compiler-7.46.0.Final.jar

org.drools.drools-mvel-7.46.0.Final.jar

org.drools.drools-mvel-compiler-7.46.0.Final.jar

org.drools.drools-mvel-parser-7.46.0.Final.jar

org.drools.drools-serialization-protobuf-7.46.0.Final.jar

org.drools.drools-templates-7.46.0.Final.jar

org.eclipse.microprofile.config.microprofile-config-api-1.4.jar

org.eclipse.microprofile.context-propagation.microprofile-context-propagation-api-1.0.1.jar

org.eclipse.microprofile.openapi.microprofile-openapi-api-1.1.2.jar

org.glassfish.jaxb.jaxb-runtime-2.3.3-b02.jar

org.glassfish.jaxb.txw2-2.3.3-b02.jar

org.graalvm.sdk.graal-sdk-20.2.0.jar

org.jboss.jandex-2.2.1.Final.jar

org.jboss.logging.jboss-logging-3.4.1.Final.jar

org.jboss.logging.jboss-logging-annotations-2.1.0.Final.jar

org.jboss.logmanager.jboss-logmanager-embedded-1.0.6.jar

org.jboss.resteasy.resteasy-core-4.5.8.Final.jar

org.jboss.resteasy.resteasy-core-spi-4.5.8.Final.jar

org.jboss.resteasy.resteasy-jackson2-provider-4.5.8.Final.jar

org.jboss.resteasy.resteasy-jaxb-provider-4.5.8.Final.jar

org.jboss.slf4j.slf4j-jboss-logmanager-1.1.0.Final.jar

org.jboss.spec.javax.ws.rs.jboss-jaxrs-api_2.1_spec-1.0.1.Final.jar

org.jboss.spec.javax.xml.bind.jboss-jaxb-api_2.3_spec-2.0.0.Final.jar

org.jboss.threads.jboss-threads-3.1.1.Final.jar

org.jetbrains.annotations-13.0.jar

org.jetbrains.kotlin.kotlin-reflect-1.4.21.jar

org.jetbrains.kotlin.kotlin-stdlib-1.4.21.jar

org.jetbrains.kotlin.kotlin-stdlib-common-1.4.21.jar

org.jetbrains.kotlin.kotlin-stdlib-jdk8-1.4.21.jar

org.jpmml.pmml-agent-1.5.1.jar

org.jpmml.pmml-model-1.5.1.jar

org.kie.kie-dmn-api-7.46.0.Final.jar

org.kie.kie-dmn-backend-7.46.0.Final.jar

org.kie.kie-dmn-core-7.46.0.Final.jar

org.kie.kie-dmn-feel-7.46.0.Final.jar

org.kie.kie-dmn-model-7.46.0.Final.jar

org.kie.kie-dmn-validation-7.46.0.Final.jar

org.kie.kie-memory-compiler-7.46.0.Final.jar

org.kie.kie-pmml-api-7.46.0.Final.jar

org.kie.kie-pmml-commons-7.46.0.Final.jar

org.kie.kie-pmml-compiler-api-7.46.0.Final-tests.jar

org.kie.kie-pmml-compiler-api-7.46.0.Final.jar

org.kie.kie-pmml-compiler-commons-7.46.0.Final.jar

org.kie.kie-pmml-compiler-core-7.46.0.Final.jar

org.kie.kie-pmml-evaluator-api-7.46.0.Final.jar

org.kie.kie-pmml-evaluator-assembler-7.46.0.Final.jar

org.kie.kie-pmml-evaluator-core-7.46.0.Final.jar

org.kie.kie-pmml-models-drools-common-7.46.0.Final.jar

org.kie.kie-pmml-models-drools-scorecard-compiler-7.46.0.Final.jar

org.kie.kie-pmml-models-drools-scorecard-evaluator-7.46.0.Final.jar

org.kie.kie-pmml-models-drools-scorecard-model-7.46.0.Final.jar

org.kie.kie-pmml-models-drools-tree-compiler-7.46.0.Final.jar

org.kie.kie-pmml-models-drools-tree-evaluator-7.46.0.Final.jar

org.kie.kie-pmml-models-drools-tree-model-7.46.0.Final.jar

org.kie.kie-pmml-models-mining-compiler-7.46.0.Final.jar

org.kie.kie-pmml-models-mining-evaluator-7.46.0.Final.jar

org.kie.kie-pmml-models-mining-model-7.46.0.Final.jar

org.kie.kie-pmml-models-regression-compiler-7.46.0.Final.jar

org.kie.kie-pmml-models-regression-evaluator-7.46.0.Final.jar

org.kie.kie-pmml-models-regression-model-7.46.0.Final.jar

org.kie.kogito.drools-compiler-1.0.0.Final.jar

org.kie.kogito.drools-core-1.0.0.Final.jar

org.kie.kogito.drools-core-static-1.0.0.Final.jar

org.kie.kogito.drools-decisiontables-1.0.0.Final.jar

org.kie.kogito.jbpm-flow-1.0.0.Final.jar

org.kie.kogito.kogito-api-1.0.0.Final.jar

org.kie.kogito.kogito-dmn-1.0.0.Final.jar

org.kie.kogito.kogito-drools-1.0.0.Final.jar

org.kie.kogito.kogito-drools-model-1.0.0.Final.jar

org.kie.kogito.kogito-internal-1.0.0.Final.jar

org.kie.kogito.kogito-pmml-1.0.0.Final.jar

org.kie.kogito.kogito-quarkus-1.0.0.Final.jar

org.kie.kogito.kogito-ruleunits-1.0.0.Final.jar

org.kie.kogito.kogito-services-1.0.0.Final.jar

org.kie.kogito.kogito-timer-1.0.0.Final.jar

org.kie.soup.kie-soup-commons-7.46.0.Final.jar

org.kie.soup.kie-soup-maven-support-7.46.0.Final.jar

org.kie.soup.kie-soup-project-datamodel-api-7.46.0.Final.jar

org.kie.soup.kie-soup-project-datamodel-commons-7.46.0.Final.jar

org.kie.soup.kie-soup-xstream-7.46.0.Final.jar

org.mvel.mvel2-2.4.10.Final.jar

org.ow2.asm.asm-9.0.jar

org.ow2.asm.asm-analysis-9.0.jar

org.ow2.asm.asm-tree-9.0.jar

org.ow2.asm.asm-util-9.0.jar

org.reactivestreams.reactive-streams-1.0.3.jar

org.slf4j.slf4j-api-1.7.30.jar

org.wildfly.common.wildfly-common-1.5.4.Final-format-001.jar

org.yaml.snakeyaml-1.27.jar

xmlpull.xmlpull-1.1.3.1.jar

xpp3.xpp3_min-1.1.4c.jar

[Tristan Radisson]

Hello,

Indeed, jar libraries should not be needed with Quarkus native runner.

How do you build your runtime image ?

Would you have a reproducer ?

Regards,

Tristan Radisson

Senior Software Quality Engineer

Business Automation Cloud

Red Hat France

trad...@redhat.com   

--
You received this message because you are subscribed to the Google Groups "Kogito development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kogito-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kogito-development/f5fe9a24-9cfe-4f94-aeb3-dbed7fda9e6an%40googlegroups.com.

[Qbyte Consulting]

PS: no surprise - one of the projects cucumber deps wasn't scoped to test.

[Qbyte Consulting]

Hi,

We use maven to build:

mvn clean package -Pnative

<profiles>
<profile>
<id>native</id>
<properties>
<failsafe.include>**/Native*IT.java</failsafe.include>
<failsafe.exclude></failsafe.exclude>
<quarkus.package.type>native</quarkus.package.type>
<!-- Native building needs a fixed port for tests -->
<tests.quarkus.http.port>8080</tests.quarkus.http.port>
</properties>
<activation>
<property>
<name>native</name>
</property>
</activation>
</profile>

...

Here are project dependncies:

<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.kie.kogito</groupId>
<artifactId>kogito-quarkus-bom</artifactId>
<version>${kogito.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>org.kie.kogito</groupId>
<artifactId>kogito-quarkus</artifactId>
<version>${kogito.version}</version>
</dependency>
<dependency>
<groupId>org.kie.kogito</groupId>
<artifactId>drools-decisiontables</artifactId>
<version>${kogito.version}</version>
</dependency>
<dependency>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-stdlib</artifactId>
<version>${kotlin.version}</version>
</dependency>

[Tristan Radisson]

Oh I see now.

You mean the native executable file is taking 55MB.
This file includes all the required information for the application to run without any extra lib/jvm, so all included by the build phase in one file. 

Indeed it can take some amount of storage (although 55MB is not that big compared to a runner jar) but will start really fast and with a slow amount of memory.

For more information on native compilation, I invite you to check Quarkus website: https://quarkus.io/guides/building-native-image

Best regards,

Tristan Radisson

Senior Software Quality Engineer

Business Automation Cloud

Red Hat France

trad...@redhat.com   

To view this discussion on the web visit https://groups.google.com/d/msgid/kogito-development/fa56d7b2-02bd-4157-8ad1-58414b1936dcn%40googlegroups.com.

[Daniele Zonca]

Hi,

One suggestion I have (if your project is a rule only project) is to use kogito-quarkus-rules extension that contains only DRL/decision table support. It has been released with Kogito 1.3 (as far as I remember) and the replacement should be very simple (see https://github.com/kiegroup/kogito-examples/blob/stable/rules-quarkus-helloworld/pom.xml#L27 )

This limits the dependency tree you have but at the same time I don't think it will have a big impact on native compilation time/jar size.

Native compilation has to navigate all the code and compile it so it takes quite a lot of time and memory. Unfortunately this cannot be incrementally done (yet?) especially because the final binary only contains reachable code but this can only be analyzed producing the final/target binary.

The limitation is at GraalVM level so we don't have any way to improve it and my personal feeling is that it will not improve soon.

I confirm that all the jars produced during a native compilation are just intermediate steps so they are not required to run.

My suggestion is to use native compilation only for pre-prod/prod envs and stay with jvm for development.

At the same time keep in mind that a problem can appear only with the native compiled artifact so it is important to check it before move it to prod (that's why I suggest this step during pre-prod/integration phase)

About test dependencies in that lib folder to be honest I don't know, maybe they are there to enable quarkus remote dev mode but I agree that looks strange. Please ask directly to quarkus team because we don't control that phase.

HTH

Daniele

To view this discussion on the web visit https://groups.google.com/d/msgid/kogito-development/CACsW2ZKbtrdbwTr9cxbaAKx5ASKNC7L6f7ebFUk_4jxvROB2ww%40mail.gmail.com.