How to get more information on using spark iceberg nessie while trying to create a database

[Ridampreet Jaggi]

Hello,

When I try to execute create database statement , I am unable to see any calls being made on the nessie side. Is there a way to enable logs or access logs to look into what calls are being received by nessie. My nessie app is deployed on EKS cluster. Any help is appreciated, thanks

[Ridampreet Jaggi]

Here is the config being set


spark = glueContext.spark_session.builder \
        .config("spark.jars.packages","org.apache.iceberg:iceberg-spark-runtime-3.5_2.12-1.5.1,org.projectnessie.nessie-integrations:nessie-spark-extensions-3.5_2.12-0.79.0") \
        .config("spark.sql.extensions", "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions") \
        .config("spark.sql.catalog.nessie.uri", url) \
        .config("spark.sql.catalog.nessie.ref", ref) \
        .config("spark.sql.catalog.nessie.authentication.type", auth_type) \
        .config("spark.sql.catalog.nessie.catalog-impl", "org.apache.iceberg.nessie.NessieCatalog") \
        .config("spark.sql.catalog.solid_dose.io-impl", "org.apache.iceberg.aws.s3.S3FileIO") \
        .config("spark.sql.catalog.nessie.warehouse", full_path_to_warehouse) \
        .config("spark.sql.catalog.nessie", "org.apache.iceberg.spark.SparkCatalog") \
        .getOrCreate()



sample sql statement to create a database, at this time we are not able to do anything with the catalog

spark.sql("CREATE DATABASE IF NOT EXISTS nessie_database").show()

[Robert Stupp]

Did you take a look at https://projectnessie.org/guides/spark-s3/#setting-up-spark-session-for-minio ?

[Ridampreet Jaggi]

Hello, 

Thank you Ajantha, Dmitri and Robert, the missing piece there was creating a namespace, I will look more into logs, will reach out if I have any questions.

Thank you so much, I was stuck on this for a long time

--
You received this message because you are subscribed to the Google Groups "projectnessie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to projectnessi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/projectnessie/013e603e-465d-47d8-bd5d-175171dc8889n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.