java.lang.RuntimeException: java.io.IOException: invalid constant type: 18

[Vikas Reddy Aravabhumi]

While running bigbench Benchmark on HDP 2.3.0.0 using ambari, the following error occurred in data generation stage

-------------------------------------------------------------------------------------------------------------------------------------------------------

JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51-2.4.5.5.el7.x86_64/jre-abrt/bin/java

/tmp/pdgfLog/1/pdgf.log:

DEBUG main pdgf.generator.BigBenchReviewGenerator - 'Clothing & Accessories_Tops & Tees' DEBUG main pdgf.generator.BigBenchReviewGenerator - 'Toys & Games_Electronics for Kids' DEBUG main pdgf.generator.BigBenchReviewGenerator - 'Toys & Games_Vehicles & Remote-Control' DEBUG main pdgf.core.dataGenerator.scheduler.DefaultPartitioner - Using default Pre-Partitioner from class pdgf.core.dataGenerator.scheduler.TemplatePartitioner 11: <generation> 84: <schema name="default"> 85: <tables> 554: <table name="product_reviews"> 555: <scheduler name="DefaultScheduler"> 556: <partitioner name="pdgf.core.dataGenerator.scheduler.TemplatePartitioner"> DEBUG main pdgf.output.FileOutputSkeleton - def path != null: '"/user/root/benchmarks/bigbench/data_refresh/"+table.getName()+"/"' && !Constants.OUTPUT_FILE_KEEP_OUTPUTDIR:false => ignoring specified <ouputDir> nodes DEBUG main pdgf.core.dataGenerator.scheduler.DefaultPartitioner - Using default Pre-Partitioner from class pdgf.core.dataGenerator.scheduler.TemplatePartitioner

11: <generation> 59: <scheduler name="DefaultScheduler"> 60: <partitioner name="pdgf.core.dataGenerator.scheduler.TemplatePartitioner" staticTableOnAllNodes="false"> DEBUG main pdgf.output.FileOutputSkeleton - def path != null: '"/user/root/benchmarks/bigbench/data_refresh/"+table.getName()+"/"' && !Constants.OUTPUT_FILE_KEEP_OUTPUTDIR:false => ignoring specified <ouputDir> nodes DEBUG main pdgf.core.dataGenerator.DataGenerator - MemoryAllocatorInterface: add Element: <schema name="bigbench"><table name="store"><field name="s_rec_start_date"><gen name="DateTimeGenerator"> WARN main pdgf.core.dataGenerator.DataGenerator - A 'pdgf.core.exceptions.ConfigurationException Exception occurred during initialization. Message: The template contains errors: java.lang.RuntimeException: java.io.IOException: invalid constant type: 18 Copy this class in an IDE of your choice to ease debugging: private class TemplateTester extends pdgf.generator.template.NextValueTemplate { public void getValue(pdgf.plugin.AbstractPDGFRandom rng,pdgf.core.dataGenerator.beans.FieldValueDTO fvdto, pdgf.core.dataGenerator.beans.GenerationContext gc) throws Exception{ fvdto.setBothValues(generator(0, rng, gc, fvdto) + " " + generator(1, rng, gc, fvdto)); } }

Location:

Location: 14: <schema name="bigbench"> 2076: <table name="store"> 2170: <field name="s_manager" primary="false" size="40" type="VARCHAR"> 2171: <gen_NullGenerator name="NullGenerator" probability="${NULL_CHANCE}"> 2172: <gen_TemplateGenerator name="TemplateGenerator"> DebugInformation: :pdgf.core.exceptions.ConfigurationException: The template contains errors:java.lang.RuntimeException: java.io.IOException: invalid constant type: 18 Copy this class in an IDE of your choice to ease debugging: private class TemplateTester extends pdgf.generator.template.NextValueTemplate { public void getValue(pdgf.plugin.AbstractPDGFRandom rng,pdgf.core.dataGenerator.beans.FieldValueDTO fvdto, pdgf.core.dataGenerator.beans.GenerationContext gc) throws Exception{ fvdto.setBothValues(generator(0, rng, gc, fvdto) + " " + generator(1, rng, gc, fvdto)); } } at pdgf.generator.template.NextValueTemplate.instance(NextValueTemplate.java:97) at pdgf.generator.TemplateGenerator.initialize(TemplateGenerator.java:102) at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:514) at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:528) at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:528) at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:528) at pdgf.core.dbSchema.Element.initStage8_initialize_(Element.java:528) at pdgf.core.dbSchema.Project.initStage8_initialize_(Project.java:722) at pdgf.core.dataGenerator.DataGenerator.initRootProject(DataGenerator.java:171) at pdgf.core.dataGenerator.DataGenerator.initialize(DataGenerator.java:139) at pdgf.core.dataGenerator.DataGenerator.start(DataGenerator.java:214) at pdgf.actions.StartAction.execute(StartAction.java:112) at pdgf.actions.ActionPrioritySortObject.execute(ActionPrioritySortObject.java:50) at pdgf.Controller.parseCmdLineArgs(Controller.java:1248) at pdgf.Controller.start(Controller.java:1385) at pdgf.Controller.main(Controller.java:1226)

[Michael Frank]

Hi,
the data generation tool PDGF uses the Javassist bytecode generation library in version 3.18.0.GA
Typically the observed error:

invalid constant type: 18

 is produced if a parser (in this case javaassist) does not understand constant type 18 (= CONSTANT_InvokeDynamic) which should only occur if javaassist encounters a class compiled  with java 1.8.
The probable cause should be a bug in javassist.
Alternatively: there is a class in PDGF's classpath which is compiled with java 1.8

Could you please check what java version is installed on your haddop Nodes (not your gateway machine)?
If you could provide the full pdgf.log file it would be very helpful. You can send it to me by mail: michael.frank(at)bankmark.de or via pastebin link.
I am mostly interested in the lines following: "Startup arguments:" more specifically the value following property: "java.runtime.version="

In the meantime i will try to reproduce your error.
One thing you could try on your side is: replace the data-generator/extlib/javassist.jar with this one: https://github.com/jboss-javassist/javassist/releases/download/rel_3_18_2_ga_build/javassist.jar
(Dont use the latest version of javassist from github 3.20.0 GA as it has an open unresolved issue which will in turn prevent it from running on  anything below java 1.8)

For completeness a link to your double post on Stackoverflow. I would like to keep the conversation in one place.
http://stackoverflow.com/questions/36051993/java-lang-runtimeexception-java-io-ioexception-invalid-constant-type-18

Cheers
Michael

[Michael Frank]

Short clarification:

I have tested it again with javassist 3.20-GA. And it works .
So you can also try replacing PDGF's javassist.jar with:  https://github.com/jboss-javassist/javassist/releases/download/rel_3_20_0_ga/javassist.jar

Please disregard this line  It does not for the tagged release version (see link above)

(Dont use the latest version of javassist from github 3.20.0 GA as it has an open unresolved issue which will in turn prevent it from running on  anything below java 1.8)

The "latest" in my statement referred to the pre-compiled javassist.jar from the root dir of the master branch ( https://github.com/jboss-javassist/javassist/blob/master/javassist.jar ) which has the mentioned issue.

Cheers,
Michael

[Vikas Reddy Aravabhumi]

Now it is working fine with java 1.7. 

@Michael: Thank you so much for your information.