Newer question

[Hahn Jiang]

Hello, 

I use voldemort as Key-Valve store. Key is string, value is identity(byte array).

I would write a builder to push data, but I cannot find voldemort in maven repository.

what version would i use in pom.xml? 

Thanks

[Felix GV]

Hi Hahn,

Sorry for the delay.

Unfortunately, Voldemort is not currently published to Maven. The process to publish to Maven Central was begun but then abandoned because of lack of time. If I remember correctly, I hit a problem with the build not generating exactly the right artifacts for the project to be accepted in OSSRH. If someone is game to figure this out, it'd be great, as I don't have the bandwidth to dive back into that now.

https://github.com/voldemort/voldemort/issues/287

That being said, you don't necessarily need to get sidetracked by this... you can also just build your identity push functionality in Voldemort proper.

Take a look at this class and the others in the same package: https://github.com/voldemort/voldemort/blob/master/contrib/hadoop-store-builder/src/java/voldemort/store/readonly/mr/AbstractHadoopStoreBuilderMapper.java

That should give you a rough idea of how JSON and Avro are currently supported with minimal duplication.

--

Felix GV
Staff Software Engineer
Data Infrastructure
LinkedIn
 
f...@linkedin.com
linkedin.com/in/felixgv

--
You received this message because you are subscribed to the Google Groups "project-voldemort" group.
To unsubscribe from this group and stop receiving emails from it, send an email to project-voldemort+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/project-voldemort.
For more options, visit https://groups.google.com/d/optout.

[Hahn Jiang]

I have written a MR Job to build index and output to hdfs, then I download the files to the machines.

Would I swap the data without VoldemortSwapJob?

Thanks

To unsubscribe from this group and stop receiving emails from it, send an email to project-voldem...@googlegroups.com.

[Felix GV]

Hi Hahn,

The VoldemortSwapJob is responsible for initiating the fetch on the storage nodes, as well as triggering the swap after the fetch is complete, so you should not have manually orchestrated the fetch beforehand if you use that. If you want to do just the swap, then that can be done via the AdminClient.

I'm not sure how you implemented the rest of your stuff, but the "cleanest" thing would probably be to leverage the VoldemortBuildAndPush job as much as possible. This would provide you with the most checks, safe guards and handshakes, some of which are merely optimizations, but some others which are fairly explicit requirements for the job to work properly.

--

Felix GV
Staff Software Engineer
Data Infrastructure
LinkedIn
 
f...@linkedin.com
linkedin.com/in/felixgv

To unsubscribe from this group and stop receiving emails from it, send an email to project-voldemort+unsubscribe@googlegroups.com.