spark.sql.catalog.nessie.warehouse

[Joe Stein]

Can this value be an s3 bucket? I tried that and I still go metastore_db created locally instead. I can create tables in s3 using LOCATION on the CREATE TABLE but files still get created if I put a path for spark.sql.catalog.nessie.warehouse. Any help is appreciated. as I need all the data on object storage (except the db that powers nessie is ok).

Thanx =8^) Joe Stein

[Robert Stupp]

Hi Joe,

this is rather an Spark question (`.warehouse` is an Spark parameter for the Iceberg/Spark catalog). But yes, that can be an S3 URL. There's documentation on how to use S3 with Iceberg on the Iceberg web site.

Robert

PS: Mind asking on our Zulip Chat? We usually respond quicker there.

On 01.12.23 03:22, Joe Stein wrote:

Can this value be an s3 bucket? I tried that and I still go metastore_db created locally instead. I can create tables in s3 using LOCATION on the CREATE TABLE but files still get created if I put a path for spark.sql.catalog.nessie.warehouse. Any help is appreciated. as I need all the data on object storage (except the db that powers nessie is ok).

Thanx =8^) Joe Stein

--
You received this message because you are subscribed to the Google Groups "projectnessie" group.
To unsubscribe from this group and stop receiving emails from it, send an email to projectnessi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/projectnessie/2d7fdeae-5637-4b91-ad38-ba9af16ec842n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Robert Stupp
@snazy