Rocksdb-Java Multiget NullPointer Exception

[udit...@gmail.com]

Hi ,

We are using the rocks-java for storing the  ordered data, and our data size is prety huge like 6TB. We do use multiGet heavily to fetch the data from rocksdb. But occasionally we are seeing a NullPointer Exception when trying to do multiGet on rocksdb. From the doc it says that if keys are missing multiGet will return list of key -> null,  but instead library throwing up NullPointer Exception. I am really not sure what could be the possible reason for behind this.

java.lang.NullPointerException: null
at java.base/java.util.Objects.requireNonNull(Objects.java:221)
at java.base/java.util.Arrays$ArrayList.<init>(Arrays.java:4323)
at java.base/java.util.Arrays.asList(Arrays.java:4310)
at org.rocksdb.RocksDB.multiGetAsList(RocksDB.java:2226)

Thanks for the help !!

[Adam Retter]

Hi,

Can you tell me which version of RocksDB please?

Thanks

> --
> You received this message because you are subscribed to the Google Groups "rocksdb" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/64953cac-c1d9-48dd-99e3-c648944a9c99n%40googlegroups.com.



--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk

[Udit Sharma]

Hi Adam,

Sorry I forgot to add that. We are using 6.11.4 version. 

Thanks

Udit

[Adam Retter]

The error is coming from here:
https://github.com/facebook/rocksdb/blob/v6.11.4/java/src/main/java/org/rocksdb/RocksDB.java#L2226

I believe you should only ever see a null if an exception is thrown
first, see: https://github.com/facebook/rocksdb/blob/v6.11.4/java/rocksjni/rocksjni.cc#L1618

Do you know how to reproduce the issue?

[Udit Sharma]

Hi Adam,

yeah, i have gone through these links. but I was more interested in what could be the possible reason for the exception in underlying rocksdb.

I can easily reproduce this on our data. but i am still trying to reproduce it independently.

Thanks

Udit

[Adam Retter]

The underlying cause should from the code imply that you have received
either an OutOfMemoryError or an ArrayIndexOutOfBounds.
If you have not, there must be something else unexpected at play...

[Udit Sharma]

Hi,

I did not receive any such exception in the app.

I do not see any sign of memory pressure on the service, and it does not crash or restart, which should have happened if there was  OutOfMemory.  Node on which it is running has 32GB RAM, from the memory tracker of rocksdb i can see that overall memory hover around 25-28 GB.

One thing I have noticed is that this is happening only for specific data which has a common prefix. I am trying to port  this data on local db, and will perform the multiget and see if i am able to replicate it that way. 

Thanks

Udit

[Udit Sharma]

Update from my side.

It happened to be an application issue, where one of the edge case was calling multiget api with 1/2 million keys, and getting the NullPointer Exception.

Thanks