Ontop-cli failure on Windows

97 views
Skip to first unread message

Johan Van Noten

unread,
Sep 21, 2022, 9:09:16 AM9/21/22
to ontop4obda
Environment
* Ontop 4.2.1 CLI
* Windows 10 Pro
* using jdbc driver for Dremio

Scenario
I wanted to perform a to-r2rml conversion, so I started the following command
c:\prog\ontop-cli\ontop.bat mapping to-r2rml -i jml.obda -p jml.properties

Problem

The CLI outputs the following:

14:50:05.993 |-INFO  in com.dremio.jdbc.Driver - Enabling Netty native memory API for Java9+
14:50:06.172 |-INFO  in c.c.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 64ms.
Base Configuration:
        - jar:file:/C:/Prog/ontop-cli/jdbc/dremio-jdbc-driver-20.1.0-202202061055110045-36733c65.jar!/sabot-default.conf

Intermediate Configuration and Plugin files, in order of precedence:
        - jar:file:/C:/Prog/ontop-cli/jdbc/dremio-jdbc-driver-20.1.0-202202061055110045-36733c65.jar!/sabot-module.conf


14:50:06.272 |-INFO  in c.o.a.arrow.memory.BaseAllocator - Debug mode disabled.
14:50:06.274 |-INFO  in c.o.a.a.m.DefaultAllocationManagerOption - allocation manager type not specified, using netty as the default type
14:50:06.275 |-INFO  in c.o.a.arrow.memory.CheckAllocator - Using DefaultAllocationManager at memory/DefaultAllocationManagerFactory.class
14:50:07.282 |-ERROR in c.c.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.  Connection: null <--> null (user client).  Closing connection.
java.lang.UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available
        at cdjd.io.netty.util.internal.PlatformDependent.directBuffer(PlatformDependent.java:490)
        at cdjd.io.netty.buffer.NettyArrowBuf.getDirectBuffer(NettyArrowBuf.java:260)
        at cdjd.io.netty.buffer.NettyArrowBuf.nioBuffer(NettyArrowBuf.java:238)
        at cdjd.io.netty.buffer.NettyArrowBuf.nioBuffers(NettyArrowBuf.java:222)
        at cdjd.io.netty.buffer.NettyArrowBuf.setBytes(NettyArrowBuf.java:382)
        at cdjd.io.netty.buffer.MutableWrappedByteBuf.setBytes(MutableWrappedByteBuf.java:373)
        at cdjd.io.netty.buffer.ExpandableByteBuf.setBytes(ExpandableByteBuf.java:27)
        at cdjd.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
        at cdjd.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
        at cdjd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
        at cdjd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
        at cdjd.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at cdjd.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at java.base/java.lang.Thread.run(Thread.java:833)
14:50:07.293 |-INFO  in c.c.dremio.sabot.rpc.user.UserClient - [USER]: Channel closed null <--> null (user client)
14:50:07.303 |-WARN  in c.c.d.telemetry.api.metrics.Metrics - Removing old metric since name matched newly registered metric. Metric name: USER-send-durations-ms
14:50:07.414 |-ERROR in c.c.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.  Connection: null <--> null (user client).  Closing connection.
java.lang.UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available
        at cdjd.io.netty.util.internal.PlatformDependent.directBuffer(PlatformDependent.java:490)
        at cdjd.io.netty.buffer.NettyArrowBuf.getDirectBuffer(NettyArrowBuf.java:260)
        at cdjd.io.netty.buffer.NettyArrowBuf.nioBuffer(NettyArrowBuf.java:238)
        at cdjd.io.netty.buffer.NettyArrowBuf.nioBuffers(NettyArrowBuf.java:222)
        at cdjd.io.netty.buffer.NettyArrowBuf.setBytes(NettyArrowBuf.java:382)
        at cdjd.io.netty.buffer.MutableWrappedByteBuf.setBytes(MutableWrappedByteBuf.java:373)
        at cdjd.io.netty.buffer.ExpandableByteBuf.setBytes(ExpandableByteBuf.java:27)
        at cdjd.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
        at cdjd.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
        at cdjd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
        at cdjd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
        at cdjd.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at cdjd.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at java.base/java.lang.Thread.run(Thread.java:833)
14:50:07.415 |-INFO  in c.c.dremio.sabot.rpc.user.UserClient - [USER]: Channel closed null <--> null (user client)
java.sql.SQLException: Failure in connecting to Dremio: cdjd.com.dremio.exec.rpc.RpcException: HANDSHAKE_COMMUNICATION : [USER]: Channel closed null <--> null (user client)
        at com.dremio.jdbc.impl.DremioExceptionMapper.map(DremioExceptionMapper.java:81)
        at com.dremio.jdbc.impl.DremioConnectionImpl.<init>(DremioConnectionImpl.java:104)
        at com.dremio.jdbc.impl.DremioJdbc41Factory.newConnection(DremioJdbc41Factory.java:72)
        at com.dremio.jdbc.impl.DremioFactory.newConnection(DremioFactory.java:67)
        at cdjd.org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
        at com.dremio.jdbc.Driver.connect(Driver.java:84)
        at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:681)
        at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:190)
        at it.unibz.inf.ontop.utils.LocalJDBCConnectionUtils.createConnection(LocalJDBCConnectionUtils.java:35)
        at it.unibz.inf.ontop.cli.OntopOBDAToR2RML.normalizeByConnectingToDB(OntopOBDAToR2RML.java:162)
        at it.unibz.inf.ontop.cli.OntopOBDAToR2RML.extractAndNormalizePPMapping(OntopOBDAToR2RML.java:137)
        at it.unibz.inf.ontop.cli.OntopOBDAToR2RML.run(OntopOBDAToR2RML.java:118)
        at it.unibz.inf.ontop.cli.Ontop.main(Ontop.java:18)
Caused by: cdjd.com.dremio.exec.rpc.RpcException: HANDSHAKE_COMMUNICATION : [USER]: Channel closed null <--> null (user client)
        at cdjd.com.dremio.exec.client.DremioClient$FutureHandler.connectionFailed(DremioClient.java:825)
        at cdjd.com.dremio.sabot.rpc.user.QueryResultHandler$ChannelClosedHandler.connectionFailed(QueryResultHandler.java:387)
        at cdjd.com.dremio.exec.rpc.BasicClient$ConnectionMultiListener$HandshakeSendListener.failed(BasicClient.java:401)
        at cdjd.com.dremio.exec.rpc.RequestIdMap$RpcListener.setException(RequestIdMap.java:193)
        at cdjd.com.dremio.exec.rpc.RequestIdMap$SetExceptionProcedure.apply(RequestIdMap.java:118)
        at cdjd.com.dremio.exec.rpc.RequestIdMap$SetExceptionProcedure.apply(RequestIdMap.java:108)
        at cdjd.com.carrotsearch.hppc.IntObjectHashMap.forEach(IntObjectHashMap.java:692)
        at cdjd.com.dremio.exec.rpc.RequestIdMap.doNotify(RequestIdMap.java:104)
        at cdjd.com.dremio.exec.rpc.RequestIdMap.channelClosed(RequestIdMap.java:71)
        at cdjd.com.dremio.exec.rpc.RemoteConnection.channelClosed(RemoteConnection.java:208)
        at cdjd.com.dremio.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete(RpcBus.java:213)
        at cdjd.com.dremio.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete(RpcBus.java:199)
        at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:577)
        at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:551)
        at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:490)
        at cdjd.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:615)
        at cdjd.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:604)
        at cdjd.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
        at cdjd.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84)
        at cdjd.io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:1158)
        at cdjd.io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:760)
        at cdjd.io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:736)
        at cdjd.io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:607)
        at cdjd.io.netty.channel.DefaultChannelPipeline$HeadContext.close(DefaultChannelPipeline.java:1352)
        at cdjd.io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:622)
        at cdjd.io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:606)
        at cdjd.io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:472)
        at cdjd.com.dremio.exec.rpc.RpcExceptionHandler.exceptionCaught(RpcExceptionHandler.java:37)
        at cdjd.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
        at cdjd.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
        at cdjd.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
        at cdjd.io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
        at cdjd.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
        at cdjd.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
        at cdjd.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
        at cdjd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:125)
        at cdjd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:174)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
        at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
        at cdjd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
        at cdjd.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at cdjd.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: cdjd.com.dremio.exec.rpc.ChannelClosedException: [USER]: Channel closed null <--> null (user client)
        at cdjd.com.dremio.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete(RpcBus.java:211)


Additional info

  • I tried in identical fashion on Ubuntu 20.04LTS and the command works fine.
  • Windows accesses the Dremio through an SSH tunnel on localhost, while Ubuntu can access it directly. I do the same for the Ontop Protégé plugin on Windows and that seems to work fine with the same properties file.

Johan Van Noten

unread,
Sep 21, 2022, 9:13:07 AM9/21/22
to ontop4obda
The Java version might be relevant.
On Windows:
openjdk 17.0.2 2022-01-18 LTS
OpenJDK Runtime Environment Corretto-17.0.2.8.1 (build 17.0.2+8-LTS)
OpenJDK 64-Bit Server VM Corretto-17.0.2.8.1 (build 17.0.2+8-LTS, mixed mode, sharing)

On Ubuntu:
openjdk version "1.8.0_342"
OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07)
OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode)

I didn't try other versions on Windows yet, so if you think it might be related, let me know and I'll try another version that should work according to your background knowledge.
Op woensdag 21 september 2022 om 15:09:16 UTC+2 schreef Johan Van Noten:

Benjamin Cogrel

unread,
Sep 23, 2022, 11:02:27 AM9/23/22
to Johan Van Noten, ontop4obda
Hi Johan,

The Dremio JDBC driver used to have issues with Java 11 (https://community.dremio.com/t/java-11-jdbc-support/5443/17?page=2).
Can you please give a try to a more recent version?

Best,
Benjamin

--
Please follow our guidelines on how to report a bug https://ontop-vkg.org/community/contributing/bug-report
---
You received this message because you are subscribed to the Google Groups "ontop4obda" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ontop4obda+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ontop4obda/d3d3a23c-1356-4340-89af-a7e8f8039a1fn%40googlegroups.com.

Johan Van Noten

unread,
Sep 23, 2022, 2:38:34 PM9/23/22
to ontop4obda
Hi Benjamin,

Thanks for your suggestion.
The newest version that I found is version 22.1.1, but trying that leads to exactly the same issue.
A fragment:
20:27:22.616 |-INFO  in com.dremio.jdbc.Driver - Enabling Netty native memory API for Java9+
20:27:22.921 |-INFO  in c.c.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 96ms.
Base Configuration:
        - jar:file:/C:/Prog/ontop-cli/jdbc/dremio-jdbc-driver-22.1.1-202208230402290397-a7010f28.jar!/sabot-default.conf


Intermediate Configuration and Plugin files, in order of precedence:
        - jar:file:/C:/Prog/ontop-cli/jdbc/dremio-jdbc-driver-22.1.1-202208230402290397-a7010f28.jar!/sabot-module.conf


20:27:23.045 |-INFO  in c.o.a.arrow.memory.BaseAllocator - Debug mode disabled.
20:27:23.049 |-INFO  in c.o.a.a.m.DefaultAllocationManagerOption - allocation manager type not specified, using netty as the default type
20:27:23.050 |-INFO  in c.o.a.arrow.memory.CheckAllocator - Using DefaultAllocationManager at memory/DefaultAllocationManagerFactory.class
20:27:24.339 |-ERROR in c.c.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.  Connection: null <--> null (user client).  Closing connection.

java.lang.UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available
(...)


But since you refer to some issues experienced with Dremio + newer Java versions in the past, I tried Java 11 anyway and that seems to solve the issue.
20:34:45.528 |-INFO  in com.dremio.jdbc.Driver - Enabling Netty native memory API for Java9+
20:34:45.815 |-INFO  in c.c.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 96ms.
Base Configuration:
        - jar:file:/C:/Prog/ontop-cli/jdbc/dremio-jdbc-driver-22.1.1-202208230402290397-a7010f28.jar!/sabot-default.conf


Intermediate Configuration and Plugin files, in order of precedence:
        - jar:file:/C:/Prog/ontop-cli/jdbc/dremio-jdbc-driver-22.1.1-202208230402290397-a7010f28.jar!/sabot-module.conf


WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by cdjd.io.netty.util.internal.ReflectionUtil (file:/C:/Prog/ontop-cli/jdbc/dremio-jdbc-driver-22.1.1-202208230402290397-a7010f28.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of cdjd.io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
20:34:45.981 |-INFO  in c.o.a.arrow.memory.BaseAllocator - Debug mode disabled.
20:34:45.984 |-INFO  in c.o.a.a.m.DefaultAllocationManagerOption - allocation manager type not specified, using netty as the default type
20:34:45.986 |-INFO  in c.o.a.arrow.memory.CheckAllocator - Using DefaultAllocationManager at memory/DefaultAllocationManagerFactory.class
20:34:52.207 |-INFO  in c.c.dremio.sabot.rpc.user.UserClient - [USER]: Channel closed null <--> null (user client)
R2RML mapping file jml-r2rml.ttl written!


Therefore, I guess that this is not a Ontop issue, but a Dremio one.
And - as you expected - Dremio is aware of it:

Stupid on my end not to have checked that first. Thanks for you pointing me to it.

Best regards,
Johan
Op vrijdag 23 september 2022 om 17:02:27 UTC+2 schreef benjami...@ontopic.ai:
Reply all
Reply to author
Forward
0 new messages