Error running the pipeline with version 0.8.0

[Sreyashi Nag]

I'm trying to run a simple gobblin pipeline to pull data from the local file system to kafka using the new 0.8.0 version. I'm getting an error that I hadn't seen before. It says that the jobconf.fullyQualifiedPath was not defined. 

SEVERE: Service JobScheduler [FAILED] has failed in the STARTING state.

java.lang.IllegalStateException: Property jobconf.fullyQualifiedPath was not defined.

at com.google.common.base.Preconditions.checkState(Preconditions.java:174)

at gobblin.scheduler.JobScheduler.startUp(JobScheduler.java:166)

at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:60)

at com.google.common.util.concurrent.Callables$3.run(Callables.java:100)

at java.lang.Thread.run(Thread.java:745)

My pull file is as follows:

job.name=Pull from local

job.group=local

job.description=A getting started example for local

source.filebased.data.directory=file:///Users/sreyashinag/Documents/test

writer.kafka.topic=local

writer.kafka.producerConfig.schemaRegistry.schema.name=local

writer.kafka.producerConfig.bootstrap.servers=localhost:9092

source.class=xyz.source

source.filebased.fs.uri=file:///

extract.table.name=abc

extract.table.type=SNAPSHOT_ONLY

extract.namespace=abc

writer.builder.class=pqr.builder

writer.destination.type=KAFKA

writer.output.format=AVRO

data.publisher.type=gobblin.publisher.NoopPublisher

writer.kafka.producerConfig.value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer

writer.kafka.producerConfig.key.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer

writer.kafka.producerConfig.schema.registry.url=http://localhost:8081

   Is there a property I'm missing out on? This pull file was sufficient when I had tested this using the older gobblin versions.

Thanks.

[Abhishek Tiwari]

This property was introduced recently, and it replaces the older 'jobconf.dir', you might need to make use of latest gobblin-standalone.properties from the release to induce backward compatibility. 

Specifically, this is relevant: 

# Directory where job configuration files are stored

jobconf.dir=${env:GOBBLIN_JOB_CONFIG_DIR}

jobconf.fullyQualifiedPath=file://${env:GOBBLIN_JOB_CONFIG_DIR}

Abhishek

--
You received this message because you are subscribed to the Google Groups "gobblin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gobblin-users+unsubscribe@googlegroups.com.
To post to this group, send email to gobbli...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gobblin-users/b80a5917-2c0d-4297-9475-72dd01feeae6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.