Jooby on Bazel

[Francesco Costa]

Hi,

I've been trying to get Jooby to run on Bazel (https://bazel.build/), it seems like something is not sitting right with some classes that are leveraged at runtime,

My build projects pretty much tries to do the following: depend on org.jooby:jooby:1.51 and org:jooby:jooby-netty

I got some error of the classloader not being able to locate a Netty related class leveraged by this statement https://github.com/jooby-project/jooby/blob/master/modules/jooby-netty/src/main/java/org/jooby/internal/netty/NettyServer.java#L361

I think the class not found was this java.lang.ClassNotFoundException: io.netty.channel.unix.UnixChannelOption, not 100% sure

I noticed that in the gradle-starter project you directly included these 2 libraries, which are not present if you generate the project in Maven using the archetype

https://github.com/jooby-project/gradle-starter/blob/master/build.gradle#L41

compile "io.netty:netty-transport-native-epoll:${dependencyManagement.importedProperties['netty.version']}:${osdetector.classifier.contains('linux') ? 'linux-x86_64' : ''}"

compile "io.netty:netty-tcnative-boringssl-static:${dependencyManagement.importedProperties['boringssl.version']}" 

Somehow I feel like the Maven build is pulling in some runtime dependencies that other build systems are not (e.g. in my bazel build I had to manually depend on guice, again I suspect Maven is much more lenient with transitive deps)

Edgar do you remember why you had to depend on those 2 classes directly for Gradle? The class that I'm missing at runtime seems to be related to those libraries, but it's unclear to me why is it even missing in the first place, nor I was able to have it pulled in from Bazel.

I'd love to be able to run Jooby on Bazel as it as much better support for protocol buffers and it's overall a much cleaner build system than any top-down alternative (mvn, gradle, sbt...)

Thanks,

[Paul Hammant]

Can you make a version of the jooby "getting started" (hello world)
application that would use Bazel rather than Maven and illustrates the
problem?

[Edgar Espina]

Hi Francesco, welcome to Jooby.

 The *-native-epoll is added to get better performance on unix/linux systems, while *boringssl is for SSL/HTTPS.

 Now both of these libraries are OS dependent, we do handle this automatically using a Maven profile.

 

 I'm not a gradle user and couldn't find a way to handle this automatically. This is why the gradle starter projects include the dependency manually.

Here is the Maven setup: https://github.com/jooby-project/jooby/blob/master/modules/jooby-netty/pom.xml#L149-L204

Hope this helps. Let me know if you have more questions.  

[Francesco Costa]

@Paul I don't have my laptop right now I will setup a little toy project and post it in case you are interested in repro

@Edgar Thanks! Ok, so this confirms you had to manually provide those for Gradle while they are auto-handle in the Maven build.

I tried to add them in but somehow the tool was not resolving them: Bazel doesn't auto handle transitive deps, so you need to point a tool to the mvn artifact and generate a dep tree.

I think I might have missed one aspect in the process: you say these are sys dependent libraries, I'm not a Maven or Gradle user so I'm only familiar to the basic concepts I noticed that you might have to specify a classifier for the OS

e.g. 

<dependency>

    <groupId>io.netty</groupId>

    <artifactId>netty-transport-native-epoll</artifactId>

    <version>4.1.32</version>

    <classifier>linux-x86_64</classifier>

</dependency>

When importing in my bazel project I passed to the gen tool something like this io.netty:netty-transport-native-epoll:4.1.32.Final which I'm suspecting is part of why it couldn't be resolved on Maven.

Edgar can you spell out for me an example for those 2 artifacts of how it gets resolved when it applies necessary subsitutions so I can at least try if it works once I pull those in? (I'm running on Linux)

Also I was trying to figure out what version of the libraries to download but I was not sure of where netty version / boringssl version is defined can you point me to those too?

Thanks!

[Edgar Espina]

Look here: http://central.maven.org/maven2/io/netty/netty-transport-native-epoll/4.1.30.Final/

You will find all the OS dependent artifacts. A linux x64 version looks like: netty-transport-native-epoll-4.1.30.Final-linux-x86_64.jar

The linux-x86_64 suffix is what you see in the Maven profile I shared before.

To know or get all transitive dependencies:

- fork/clone https://github.com/jooby-project/hello-starter

- then run: mvn clean dependency:tree

Cool?

--
You received this message because you are subscribed to the Google Groups "jooby-project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooby-projec...@googlegroups.com.
To post to this group, send email to jooby-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jooby-project/24438a01-a421-49ec-80de-6dc093f653f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

edgar

[Fra]

Thanks Edgar and Paul, I will try later on today.

[Francesco Costa]

Hi Edgar,

I still couldn't make it run on Bazel, here some details:

I started by creating a jooby hello world following your doc

mvn archetype:generate -B -DgroupId=com.mycompany -DartifactId=my-app -Dversion=1.0-SNAPSHOT -DarchetypeArtifactId=jooby-archetype -DarchetypeGroupId=org.jooby -DarchetypeVersion=1.5.1

cd my-app

mvn jooby:run

@:~/Desktop/work/API/my-app$ tree .

.

├── conf

│   ├── application.conf

│   └── logback.xml

├── pom.xml

├── public

└── src

    ├── etc

    │   └── stork.yml

    ├── main

    │   └── java

    │       └── com

    │           └── mycompany

    │               └── App.java

    └── test

        └── java

            └── com

                └── mycompany

                    └── AppTest.java

Then as you suggested I extracted the dep list 

@:~/Desktop/work/API/my-app$ mvn dependency:tree

[INFO] com.mycompany:my-app:jar:1.0-SNAPSHOT

[INFO] +- org.jooby:jooby-netty:jar:1.5.1:compile

[INFO] |  +- org.jooby:jooby:jar:1.5.1:compile

[INFO] |  |  +- com.typesafe:config:jar:1.3.3:compile

[INFO] |  |  +- org.jooby:funzy:jar:0.1.0:compile

[INFO] |  |  +- com.google.guava:guava:jar:25.1-jre:compile

[INFO] |  |  |  +- com.google.code.findbugs:jsr305:jar:3.0.2:compile

[INFO] |  |  |  +- org.checkerframework:checker-qual:jar:2.0.0:compile

[INFO] |  |  |  +- com.google.errorprone:error_prone_annotations:jar:2.1.3:compile

[INFO] |  |  |  +- com.google.j2objc:j2objc-annotations:jar:1.1:compile

[INFO] |  |  |  \- org.codehaus.mojo:animal-sniffer-annotations:jar:1.14:compile

[INFO] |  |  \- com.google.inject.extensions:guice-multibindings:jar:4.2.0:compile

[INFO] |  |     \- com.google.inject:guice:jar:4.2.0:compile

[INFO] |  |        +- javax.inject:javax.inject:jar:1:compile

[INFO] |  |        \- aopalliance:aopalliance:jar:1.0:compile

[INFO] |  +- io.netty:netty-transport:jar:4.1.27.Final:compile

[INFO] |  |  \- io.netty:netty-buffer:jar:4.1.27.Final:compile

[INFO] |  +- io.netty:netty-resolver:jar:4.1.27.Final:compile

[INFO] |  +- io.netty:netty-codec:jar:4.1.27.Final:compile

[INFO] |  +- io.netty:netty-codec-http:jar:4.1.27.Final:compile

[INFO] |  +- io.netty:netty-codec-http2:jar:4.1.27.Final:compile

[INFO] |  +- io.netty:netty-common:jar:4.1.27.Final:compile

[INFO] |  +- io.netty:netty-handler:jar:4.1.27.Final:compile

[INFO] |  +- org.javassist:javassist:jar:3.22.0-GA:compile

[INFO] |  +- io.netty:netty-transport-native-epoll:jar:4.1.27.Final:compile

[INFO] |  |  \- io.netty:netty-transport-native-unix-common:jar:4.1.27.Final:compile

[INFO] |  +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.27.Final:compile

[INFO] |  \- io.netty:netty-tcnative-boringssl-static:jar:linux-x86_64:2.0.8.Final:compile

[INFO] +- ch.qos.logback:logback-classic:jar:1.2.3:compile

[INFO] |  +- ch.qos.logback:logback-core:jar:1.2.3:compile

[INFO] |  \- org.slf4j:slf4j-api:jar:1.7.25:compile

[INFO] +- junit:junit:jar:4.12:test

[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test

[INFO] \- io.rest-assured:rest-assured:jar:3.1.0:test

[INFO]    +- org.codehaus.groovy:groovy:jar:2.5.0:test

[INFO]    +- org.codehaus.groovy:groovy-xml:jar:2.4.12:test

[INFO]    +- org.apache.httpcomponents:httpclient:jar:4.5.5:test

[INFO]    |  +- org.apache.httpcomponents:httpcore:jar:4.4.9:test

[INFO]    |  +- commons-logging:commons-logging:jar:1.2:test

[INFO]    |  \- commons-codec:commons-codec:jar:1.10:test

[INFO]    +- org.apache.httpcomponents:httpmime:jar:4.5.5:test

[INFO]    +- org.hamcrest:hamcrest-library:jar:1.3:test

[INFO]    +- org.ccil.cowan.tagsoup:tagsoup:jar:1.2.1:test

[INFO]    +- io.rest-assured:json-path:jar:3.1.0:test

[INFO]    |  +- org.codehaus.groovy:groovy-json:jar:2.4.12:test

[INFO]    |  \- io.rest-assured:rest-assured-common:jar:3.1.0:test

[INFO]    \- io.rest-assured:xml-path:jar:3.1.0:test

[INFO]       +- org.apache.commons:commons-lang3:jar:3.7:test

[INFO]       \- javax.xml.bind:jaxb-api:jar:2.2.12:test

Then I looked at the main pom.xml

 <parent>

    <groupId>org.jooby</groupId>

    <artifactId>modules</artifactId>

    <version>1.5.1</version>

  </parent>

  <artifactId>my-app</artifactId>

  <groupId>com.mycompany</groupId>

  <version>1.0-SNAPSHOT</version>

  <name>my-app</name>

  <description>generated by Jooby archetype</description>

  <properties>

    <jooby.version>1.5.1</jooby.version>

    <!-- Startup class -->

    <application.class>com.mycompany.App</application.class>

  </properties>

  <dependencies>

    <!-- Server -->

    <dependency>

      <groupId>org.jooby</groupId>

      <artifactId>jooby-netty</artifactId>

    </dependency>

    <!-- logging -->

    <dependency>

      <groupId>ch.qos.logback</groupId>

      <artifactId>logback-classic</artifactId>

    </dependency>

    <!-- Tests -->

    <dependency>

      <groupId>junit</groupId>

      <artifactId>junit</artifactId>

      <scope>test</scope>

    </dependency>

    <dependency>

      <groupId>io.rest-assured</groupId>

      <artifactId>rest-assured</artifactId>

      <scope>test</scope>

    </dependency>

  </dependencies>

here we have 2 ways of moving foreward

GenerateWorkspace 

  https://github.com/bazelbuild/migration-tooling 

  https://docs.bazel.build/versions/master/generate-workspace.html

BazelDeps 

  https://github.com/johnynek/bazel-deps

I tried both:

GenerateWorkspace

You can check the code out here

https://github.com/frabass/joobyOnBazelGenerateWorkspace

This is how I generated the dependencies.bzl

@: ~/Desktop/work/migration-tooling/generate_workspace$ bazel run //generate_workspace -- --maven_project=~/Desktop/work/API/my-app --repositories=http://uk.maven.org/maven2 -o ~/Desktop/work/API

This is the entire deps set generated:

@: ~/Desktop/work/API/test$ bazel query '//thirdparty:all' 

INFO: Invocation ID: b4ea2d84-42d0-4352-8120-10c5952229bc

//thirdparty:org_jooby_jooby_netty

//thirdparty:org_jooby_jooby

//thirdparty:org_jooby_funzy

//thirdparty:org_javassist_javassist

//thirdparty:junit_junit

//thirdparty:io_rest_assured_rest_assured

//thirdparty:org_hamcrest_hamcrest_library

//thirdparty:org_hamcrest_hamcrest_core

//thirdparty:org_apache_httpcomponents_httpmime

//thirdparty:org_apache_httpcomponents_httpclient

//thirdparty:org_apache_httpcomponents_httpcore

//thirdparty:io_rest_assured_xml_path

//thirdparty:org_codehaus_groovy_groovy_xml

//thirdparty:org_ccil_cowan_tagsoup_tagsoup

//thirdparty:javax_xml_bind_jaxb_api

//thirdparty:io_rest_assured_json_path

//thirdparty:org_codehaus_groovy_groovy_json

//thirdparty:io_rest_assured_rest_assured_common

//thirdparty:org_codehaus_groovy_groovy

//thirdparty:org_apache_commons_commons_lang3

//thirdparty:io_netty_netty_transport_native_epoll

//thirdparty:io_netty_netty_transport

//thirdparty:io_netty_netty_resolver

//thirdparty:io_netty_netty_handler

//thirdparty:io_netty_netty_common

//thirdparty:io_netty_netty_codec_http2

//thirdparty:io_netty_netty_codec_http

//thirdparty:io_netty_netty_codec

//thirdparty:commons_logging_commons_logging

//thirdparty:commons_codec_commons_codec

//thirdparty:com_typesafe_config

//thirdparty:com_google_inject_extensions_guice_multibindings

//thirdparty:com_google_inject_guice

//thirdparty:javax_inject_javax_inject

//thirdparty:com_google_guava_guava

//thirdparty:ch_qos_logback_logback_classic

//thirdparty:org_slf4j_slf4j_api

//thirdparty:ch_qos_logback_logback_core

//thirdparty:aopalliance_aopalliance

I compared with the mvn dep tree output looked reasonable

Then I setup the app to depend on jooby and jooby-netty, I tried to build run and I was missing some compile time deps. this is the minimum set of dependency to build the app correctly

        #    "//thirdparty:org_jooby_jooby_netty",

        #    "//thirdparty:org_jooby_jooby",

        #    "//thirdparty:com_google_inject_guice",

        #    "//thirdparty:org_jooby_funzy",

to build bazel bulid app

to build/run bazel run app

If you run it this is the error that you get

bazel run app

INFO: Invocation ID: 736f44c2-5dc8-4421-9ce2-e1269fa5099c

INFO: Analysed target //:app (1 packages loaded, 2 targets configured).

INFO: Found 1 target...

Target //:app up-to-date:

  bazel-bin/app.jar

  bazel-bin/app

INFO: Elapsed time: 0.128s, Critical Path: 0.00s

INFO: 0 processes.

INFO: Build completed successfully, 1 total action

INFO: Build completed successfully, 1 total action

20:39:50.100 [main] DEBUG com.mycompany.App - config tree:

└── merge of system properties

 └── org/jooby/spi/server.conf @ jar:file:/home/fra/.cache/bazel/_bazel_fra/d3c68d0a33a12fb71db8db895996d09e/external/org_jooby_jooby_netty/org/jooby/jooby-netty/1.5.1/jooby-netty-1.5.1.jar!/org/jooby/spi/server.conf

  └── org/jooby/mime.properties @ jar:file:/home/fra/.cache/bazel/_bazel_fra/d3c68d0a33a12fb71db8db895996d09e/external/org_jooby_jooby/org/jooby/jooby/1.5.1/jooby-1.5.1.jar!/org/jooby/mime.properties

   └── org/jooby/jooby.conf @ jar:file:/home/fra/.cache/bazel/_bazel_fra/d3c68d0a33a12fb71db8db895996d09e/external/org_jooby_jooby/org/jooby/jooby/1.5.1/jooby-1.5.1.jar!/org/jooby/jooby.conf: 1

20:39:50.109 [main] DEBUG org.jooby.spi.Server - netty.threads.boss(1)

20:39:50.111 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework

20:39:50.114 [main] DEBUG io.netty.channel.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 16

20:39:50.124 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false

20:39:50.124 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512

20:39:50.133 [main] DEBUG io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false

20:39:50.134 [main] DEBUG io.netty.util.internal.PlatformDependent0 - Java version: 8

20:39:50.134 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available

20:39:50.134 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available

20:39:50.135 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available

20:39:50.135 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: available

20:39:50.135 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true

20:39:50.135 [main] DEBUG io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9

20:39:50.135 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.DirectByteBuffer.<init>(long, int): available

20:39:50.135 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available

20:39:50.135 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp/mycompany

20:39:50.135 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)

20:39:50.136 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false

20:39:50.136 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.maxDirectMemory: 3679977472 bytes

20:39:50.136 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1

20:39:50.136 [main] DEBUG io.netty.util.internal.CleanerJava6 - java.nio.ByteBuffer.cleaner(): available

20:39:50.140 [main] DEBUG io.netty.util.internal.PlatformDependent - org.jctools-core.MpscChunkedArrayQueue: available

20:39:50.143 [main] DEBUG org.jooby.spi.Server - netty.threads.worker(16)

20:39:50.156 [main] INFO com.mycompany.App - Stopped

20:39:50.157 [main] ERROR com.mycompany.App - An error occurred while starting the application:

java.lang.NoClassDefFoundError: io/netty/channel/unix/UnixChannelOption

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:763)

at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)

at java.net.URLClassLoader.access$100(URLClassLoader.java:74)

at java.net.URLClassLoader$1.run(URLClassLoader.java:369)

at java.net.URLClassLoader$1.run(URLClassLoader.java:363)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:362)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at org.jooby.internal.netty.NettyServer.findOption(NettyServer.java:361)

at org.jooby.internal.netty.NettyServer.lambda$configure$2(NettyServer.java:334)

at java.lang.Iterable.forEach(Iterable.java:75)

at org.jooby.internal.netty.NettyServer.configure(NettyServer.java:333)

at org.jooby.internal.netty.NettyServer.bootstrap(NettyServer.java:299)

at org.jooby.internal.netty.NettyServer.start(NettyServer.java:280)

at org.jooby.Jooby.start(Jooby.java:2236)

at org.jooby.Jooby.start(Jooby.java:2178)

at org.jooby.Jooby.run(Jooby.java:2107)

at com.mycompany.App.main(App.java:15)

Caused by: java.lang.ClassNotFoundException: io.netty.channel.unix.UnixChannelOption

at java.net.URLClassLoader.findClass(URLClassLoader.java:382)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 22 common frames omitted

20:39:52.365 [globalEventExecutor-2-1] DEBUG io.netty.util.internal.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024

20:39:52.366 [globalEventExecutor-2-1] DEBUG io.netty.util.internal.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096

I tried to add all the deps in the main app :lib target so that they would all be pulled in regardless transitivity, same thing, build fine, run error.

BazelDeps 

Here's the code

https://github.com/frabass/joobyOnBazelWithBazelDeps

Very similar stuff

I setup my dependency.yaml with jooby and jooby-netty and generated the transitive closure

options:
	buildHeader: []
	languages: [ "java" ]
	resolverType: "coursier"
	resolvers:
	- id: "mavencentral"
	type: "default"
	url: https://repo.maven.apache.org/maven2/
	strictVisibility: false
	transitivity: runtime_deps
	versionConflictPolicy: highest
	thirdPartyDirectory: thirdparty
	dependencies:
	org.jooby:
	jooby:
	lang: java
	modules: ["", "netty"]
	version: "1.5.1"

This is how I generated the dependencies.bzl

@:~/Desktop/work/bazel-deps$ bazel run //:parse -- generate -r ~/Desktop/work/API/jooby-bazel-starter/ -s thirdparty/workspace.bzl -d dependencies.yaml 

This is the entire deps set generated, the tool has a slightly different format, but same stuff

@:~/Desktop/work/API/jooby-bazel-starter$ bazel query '//thirdparty/...'

INFO: Invocation ID: 7d428e86-4780-4b0c-a59f-d7c2c527872c

//thirdparty/org/jooby:jooby_netty

//thirdparty/org/jooby:jooby

//thirdparty/org/jooby:funzy

//thirdparty/org/javassist:javassist

//thirdparty/io/netty:netty_transport_native_epoll_jar_linux_x86_64

//thirdparty/io/netty:netty_tcnative_boringssl_static_jar_linux_x86_64

//thirdparty/io/netty:netty_codec_http2

//thirdparty/io/netty:netty_handler

//thirdparty/io/netty:netty_codec_http

//thirdparty/io/netty:netty_codec

//thirdparty/io/netty:netty_transport

//thirdparty/io/netty:netty_resolver

//thirdparty/io/netty:netty_buffer

//thirdparty/io/netty:netty_common

//thirdparty/com/typesafe:config

//thirdparty/com/google/inject/extensions:guice_multibindings

//thirdparty/com/google/inject:guice

//thirdparty/javax/inject:javax_inject

//thirdparty/com/google/guava:guava

//thirdparty/org/codehaus/mojo:animal_sniffer_annotations

//thirdparty/org/checkerframework:checker_qual

//thirdparty/com/google/j2objc:j2objc_annotations

//thirdparty/com/google/errorprone:error_prone_annotations

//thirdparty/com/google/code/findbugs:jsr305

//thirdparty/ch/qos/logback:logback_classic

//thirdparty/org/slf4j:slf4j_api

//thirdparty/ch/qos/logback:logback_core

//thirdparty/aopalliance:aopalliance

same commands to build and run

had to had the same base targets to be able to build successfully

        #    "//thirdparty:org_jooby_jooby_netty",

        #    "//thirdparty:org_jooby_jooby",

        #    "//thirdparty:com_google_inject_guice",

        #    "//thirdparty:org_jooby_funzy",

 

Same exact exception on run

Did the same test of adding in all the deps in the target, still class not found

I looked closely at all the netty deps in the various lists, I can't really spot anything odd and I don't know in which package io.netty.channel.unix.UnixChannelOption is shipped.

I saw that on the netty repo they have a netty-all package so I'm going to try to add that and see if it works...

The last thing that I can try is to package a fat jar from your hello world maven project and try to depend on that in the bazel one and see if it works...

I'm running out of ideas if you have any suggestion is appreciated.

Thanks,

[Francesco Costa]

It worked:

I tested it on the GenerateWorkspace project

I modified the WORKSPACE file to pull in netty-all

load("//:generate_workspace.bzl", "generated_maven_jars")

generated_maven_jars()

maven_jar(

    name = "netty_all",

    artifact = "io.netty:netty-all:jar:4.1.32.Final",

)

maven_server(

    name = "maven_uk_server",

    url = "http://uk.maven.org/maven2",

)

And added it to the minmal set into the main BUILD

package(default_visibility = ["//visibility:public"])

java_library(

    name = "lib",

    srcs = glob(["src/main/java/com/mycompany/*.java"]),

    deps = [

        "@netty_all//jar",

        "//thirdparty:org_jooby_jooby_netty",

        "//thirdparty:org_jooby_jooby",

        "//thirdparty:com_google_inject_guice",

        "//thirdparty:org_jooby_funzy",

    ],

)

java_binary(

    name = "app",

    main_class = "com.mycompany.App",

    runtime_deps = [":lib"]

)

I definitely want to be able to clean this up so someone else can use it as well is there a way that I can track down what netty maven package contains io.netty.channel.unix.UnixChannelOption

I feel like if I can add that it should work as well and we can spare pulling in extra deps.

Thanks,

[Francesco Costa]

Found the missing package

@:~/Desktop/work/API/my-app$ mvn clean dependency:copy-dependencies -D=target/temp

@:~/Desktop/work/API/my-app$ for j in target/dependency/*.jar; do /usr/lib/jvm/jdk-11.0.1/bin/jar -tf $j | grep io.netty.channel.unix.UnixChannelOption && echo $j; done

io/netty/channel/unix/UnixChannelOption.class

target/dependency/netty-transport-native-unix-common-4.1.27.Final.jar

And if anyone is interested here's the working starter, I need to re add test and everything I stripped and a couple sentences on how to generate that stuff, but it seems to run!

https://github.com/frabass/joobyOnBazelWithBazelDeps

Thanks for your help Edgar, great framework!

[Edgar Espina]

Cool! Wonder why bazel doesn't resolve transitive dependencies?

Another (probably easier alternative) is to replace jooby-netty (and all the netty-* deps) with jooby-undertow. Undertow it is a single dep + xnio (which I think it is 2 or 3 more jars)

--

You received this message because you are subscribed to the Google Groups "jooby-project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooby-projec...@googlegroups.com.
To post to this group, send email to jooby-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jooby-project/2d73f61d-555b-4e02-bdba-55fb5399c9cf%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

edgar

[Francesco Costa]

I'm not sure if I fully got your question:

If you are talking about resolving the specific dependency that I was missing I am not sure: both tools use some sort of library to resolve the transitive graph from the artifact ids, that library was not pulled in by neither (probably cause they are based on the same code), the parent one was definitely resolved though, it might have been something to do on how is declared attached in Maven, again I'm not a Maven expert so I'm not sure.

[INFO] |  +- io.netty:netty-transport-native-epoll:jar:4.1.27.Final:compile

[INFO] |  |  \- io.netty:netty-transport-native-unix-common:jar:4.1.27.Final:compile

If you are talking about in general why Bazel does not support transitive dependencies out of the box I can explain that: Bazel is the external version of the build tool used at Google called Blaze.

The philosophy is to call out your dependencies and divide your codebase in functional block of reasonable size to end up with a much lighter more efficient and acyclic deps graph.

More info here: https://bazel.build/faq.html

The performance improvement is dramatic, you can find a lot of stories of big company moving from 60min to 5min build time

https://redfin.engineering/we-switched-from-maven-to-bazel-and-builds-got-10x-faster-b265a7845854

https://www.youtube.com/watch?v=nKx21CsT744

The model works extremely well on large code bases where most of your code is built in house and everything is compiled from head. In the case of an hobby project or a smaller project were you are depending on external libraries for the most part it can be painful. By default Bazel can depend on external jars fetched from maven central, but as they are, no transitive deps, this is intentional, to keep the deps graph small.

Personally, I found Bazel to be one of the best build systems around even if you don't have a codebase the size of Google. It's literally the only one that works bottom up rather than top down. Every other system mvn,sbt,gradle etc.. (I tried a lot of them) build everything in their path avalanche-style, makes it really high burden to create compilation modules (in some systems you literally have to do subprojects, which then mess up the IDE).

Bazel/Blaze is so awesome that an ex-googler who moved to Twitter implemented Pants, which is essentially blaze and twitter moved to that system.

It was painful to figure out how to get all the Jooby deps to behave, and I will probably have similar problems for every module / jar I pull in (it's pretty straightforward for compile times deps, it's a little more subtle with runtime deps) but I think it'll pay off :)

[Francesco Costa]

And I forgot... Bazel supports protocol buffers the way they should be supported https://docs.bazel.build/versions/master/be/java.html#java_proto_library

I tried several plugins on mvn and gradle, they were all quite awful, forcing to group all your protos in a single place

[Edgar Espina]

Thanks for explanation! Appreciate it.

--
You received this message because you are subscribed to the Google Groups "jooby-project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooby-projec...@googlegroups.com.
To post to this group, send email to jooby-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jooby-project/262c4abd-1d01-44f8-bf11-f037a3347a0f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

edgar