SEVERE [io.qua.ope.run.exp.otl.VertxGrpcExporter] (vert.x-eventloop-thread-4) Failed to export spans.

[Vikram Roopchand]

Hi There,

Hope everyone is doing well.

Upgraded to the latest Nessie (docker image) and to iceberg-spark-runtime-3.5_2.12-1.4.2.jar

Started getting this exception (no code changes done at the client end).

org.projectnessie.error.NessieBadRequestException: Bad Request (HTTP/400): Key 'SMARTSHIPANALYST_TESTVIKRAM01' already exists with content ID 6b07c753-b996-4907-a7d7-8863c0dadb1f, which is different from the content ID 43dc09f5-e221-421e-92d2-6b68f9716898 in the operation
at org.projectnessie.error.ErrorCode.lambda$asException$1(ErrorCode.java:66) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at java.util.Optional.map(Optional.java:265) ~[?:?]
at org.projectnessie.error.ErrorCode.asException(ErrorCode.java:66) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.projectnessie.client.rest.ResponseCheckFilter.checkResponse(ResponseCheckFilter.java:68) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.projectnessie.client.rest.NessieHttpResponseFilter.filter(NessieHttpResponseFilter.java:29) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.projectnessie.client.http.impl.jdk11.JavaRequest.lambda$executeRequest$1(JavaRequest.java:140) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at java.util.ArrayList.forEach(ArrayList.java:1541) ~[?:?]
at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1085) ~[?:?]
at org.projectnessie.client.http.impl.jdk11.JavaRequest.executeRequest(JavaRequest.java:140) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.projectnessie.client.http.HttpRequest.post(HttpRequest.java:90) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.projectnessie.client.rest.v1.RestV1TreeClient.commitMultipleOperations(RestV1TreeClient.java:204) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.projectnessie.client.rest.v1.RestV1Client$ExceptionRewriter.invoke(RestV1Client.java:78) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at com.sun.proxy.$Proxy58.commitMultipleOperations(Unknown Source) ~[?:?]
at org.projectnessie.client.rest.v1.HttpCommitMultipleOperations.commit(HttpCommitMultipleOperations.java:34) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.apache.iceberg.nessie.NessieIcebergClient.commitTable(NessieIcebergClient.java:483) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.apache.iceberg.nessie.NessieTableOperations.doCommit(NessieTableOperations.java:141) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:135) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]
at org.apache.iceberg.SnapshotProducer.lambda$commit$2(SnapshotProducer.java:400) ~[iceberg-spark-runtime-3.5_2.12-1.4.2.jar:?]

And this on the Nessie Docker console:

23-12-26 07:33:52,592 INFO  [io.qua.htt.access-log] (executor-thread-1) 127.0.0.1 - - [26/Dec/2023:07:33:52 +0000] "POST /api/v1/contents?ref=main&hashOnRef=4560d70a4eea6b927998fefb058890a600ec010df3839f9a74c256e9b47ad770 HTTP/1.1" 200 22
2023-12-26 07:33:53,974 SEVERE [io.qua.ope.run.exp.otl.VertxGrpcExporter] (vert.x-eventloop-thread-4) Failed to export spans. The request could not be executed. Full error message: Connection refused: localhost/127.0.0.1:4317

Nessie:

 sudo docker run --network "host" -p 19120:19120 -e JAVA_OPTS_APPEND="-Dnessie.version.store.type=jdbc -Dquarkus.datasource.jdbc.url=jdbc:postgresql://xxx:yyy/phantom -Dquarkus.datasource.username=phantom -Dquarkus.datasource.password=xxx" ghcr.io/projectnessie/nessie

Could any one assist ? Would be very grateful.

best regards,

Vikram

[Alex Dutra]

Hi Vikram,

Thank you for your interest in Nessie!

The first error (NessieBadRequestException) happens because apparently you tried to update an existing table, but its content ID did not match. If you can share the details of your REST requests, maybe we'll be able to help you further.

The second error (Failed to export spans) is nothing to worry about: it's logged when telemetry has not been configured, and so Nessie cannot export spans because it's trying to contact the default telemetry collector at localhost:4317. If you don't need telemetry at all, you can disable it by adding this environment variable:

-e QUARKUS_OTEL_SDK_DISABLED=true 

Lastly, may I suggest that you use our Zulip chat instead of this mailing list for all future questions:

https://project-nessie.zulipchat.com

Thanks!

Alex

[Vikram Roopchand]

Dear Alex,

Thank you for such a quick response. Much appreciated.

The first error happens via iceberg. When we are performing a "insert into with a select from" a table from a branch.

Something on the following lines "insert into deepinsight.SMARTSHIPANALYST_TESTVIKRAM01 select (`__Dmx_Internal_ID__` + 0),Name_,Age,Timestamp,Id from deepinsight.`SMARTSHIPANALYST_TESTVIKRAM01_20231227125559@DI_ETL_20231227125559_SMARTSHIPANALYST_TESTVIKRAM01`"

We are not using the REST APIs. 

A branch was created, and then a new table created on that branch, data inserted to this new table via parquet and then a insert into via select happening ... Please let  me know how I can help you debug this further.

Will move to Zulip chat here on.

best regards,

Vikram

--
You received this message because you are subscribed to the Google Groups "projectnessie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to projectnessi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/projectnessie/1aa78870-abde-4ad0-a3b1-cf8d3cf6d4e7n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.