H2 Shell stuck wanting for response from database running mixed mode (1.4.196)

[Scott Huddleston]

Hello,

I have an application that uses H2 1.4.196 running in mixed mode.  Recently, I started the H2 Shell and instead of seeing the prompt like normal, it just hangs.

The application uses the following url to connect

jdbc:h2:file:/var/packet-manager/data/database/packetmanager;MVCC=TRUE;AUTO_SERVER=TRUE;IFEXISTS=TRUE

After starting the application, the lock file packetmanager.lock.db contains

/var/packet-manager/data/database $ more packetmanager.lock.db
#FileLock
#Tue Nov 28 19:14:03 GMT 2023
server=172.20.0.2\:45062
hostName=19e5a52a4778
method=file
id=18c175a4d0dd653a64b0d292608b53c9345d636ec1c

Below is the output from the H2 Shell when executed in the same container and with same user as the application.  Trace level set to 3.

/usr/packet-manager/tools $ java -cp h2-1.4.196.jar org.h2.tools.Shell -url "jdbc:h2:file:/var/packet-manager/data/database/packetmanager;MVCC=TRUE;AUTO_SERVER=TRUE;IFEXISTS=TRUE;TRACE_LEVEL_SYSTEM_OUT=3" -user <username> -password <password>

2023-11-28 20:36:17 database: opening /var/packet-manager/data/database/packetmanager (build 196)
2023-11-28 20:36:17 fileLock: load {server=172.20.0.2:45062, hostName=19e5a52a4778, method=file, id=18c175a4d0dd653a64b0d292608b53c9345d636ec1c}

You can see that the H2 Shell loads the existing lock file, the database is open and a server was started on 172.20.0.2:45062.  After that there is no further output and it just hangs, so I took a thred dump.

"main" #1 prio=5 os_prio=0 cpu=256.68ms elapsed=4.09s tid=0x00007f9d504ba800 nid=0x81 runnable  [0x00007f9d50649000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(java...@11.0.20.1/Native Method)
        at java.net.SocketInputStream.socketRead(java...@11.0.20.1/Unknown Source)
        at java.net.SocketInputStream.read(java...@11.0.20.1/Unknown Source)
        at java.net.SocketInputStream.read(java...@11.0.20.1/Unknown Source)
        at java.io.BufferedInputStream.fill(java...@11.0.20.1/Unknown Source)
        at java.io.BufferedInputStream.read(java...@11.0.20.1/Unknown Source)
        - locked <0x00000000d2bf4098> (a java.io.BufferedInputStream)
        at java.io.DataInputStream.readInt(java...@11.0.20.1/Unknown Source)
        at org.h2.value.Transfer.readInt(Transfer.java:153)
        at org.h2.store.FileLock.checkServer(FileLock.java:247)
        at org.h2.store.FileLock.lock(FileLock.java:129)
        - locked <0x00000000d2b72798> (a org.h2.store.FileLock)
        at org.h2.engine.Database.open(Database.java:662)
        - locked <0x00000000d2c4ecc8> (a org.h2.engine.Database)
        at org.h2.engine.Database.openDatabase(Database.java:276)
        at org.h2.engine.Database.<init>(Database.java:270)
        at org.h2.engine.Engine.openSession(Engine.java:64)
        at org.h2.engine.Engine.openSession(Engine.java:176)
        - locked <0x00000000d2d8da18> (a org.h2.engine.Engine)
        at org.h2.engine.Engine.createSessionAndValidate(Engine.java:154)
        at org.h2.engine.Engine.createSession(Engine.java:137)
        at org.h2.engine.Engine.createSession(Engine.java:27)
        at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:354)
        at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:116)
        at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:100)
        at org.h2.Driver.connect(Driver.java:69)
        at java.sql.DriverManager.getConnection(java...@11.0.20.1/Unknown Source)
        at java.sql.DriverManager.getConnection(java...@11.0.20.1/Unknown Source)
        at org.h2.tools.Shell.runTool(Shell.java:148)
        at org.h2.tools.Shell.main(Shell.java:81)

It looks like the H2 Shell is waiting on a response from the server as initiated by FileLock.checkServer, but never receives one?  

Some details about the application

Environment: docker container using Bellsoft Liberica Java 11 image (Alpine Linux base image) (bellsoft/liberica-openjre-alpine-musl:11.0.20.1-1-x86_64)
Docker Engine version: 19.03.2 Community
Springboot version: 2.5.5
H2 version: 1.4.196

And some notes

- If I stop the application and execute the H2 Shell I'm able to connect without issue.
- I have a few more applications using H2 with the exact same setup.  I'm able to connect via the H2 Shell without issue on those.

I've looked through both Stack Overflow and Groups with various keywords with no luck.

Does anyone have any suggestions on how to solve this issue?

Thanks,

Scott

[Noel Grandin]

On 11/29/2023 12:42 AM, Scott Huddleston wrote:
>

Where there other threads running when you took that thread dump? Because I dont' see the actual server thread running,
which it what I would have expected.

So most likely what happened is that some other program is running on the port that H2 wants to use, the actual H2 TCP
server failed to start, the console is now connecting to that other program, and the other program is not sending back
the data that the console client expects.

[Scott Huddleston]

Sorry, that thread dump was just an excerpt from the H2 shell.  Other threads were definitely running.  I've attached a thread dump from the application that runs H2 and the H2 shell.

I didn't see any threads named "H2 TCP Server" in the application thread dump, so your comment about not starting the TCP server seems spot on.  Is "H2 TCP Server" the correct thread name to look for?

Thanks for the quick response.

[Noel Grandin]

On Wed, 29 Nov 2023 at 17:46, Scott Huddleston <talascend...@gmail.com> wrote:

I don't see any threads named "H2 TCP Server" in the application thread dump, so your comment about the TCP server failing to start seems spot on.  Is that the correct name of the thread to look for?

Yes, that would be the one.