SqlToNosql problem with ElasticSearch

[Cristian Mihai Barca]

Hi,

I have configured the import.properties, data-config.xml and copied in /lib all the required libraries.

import.properties:
#common properties
sql-data-config-file=data-config.xml
autoCommitSize=500
#available values are mongo,es,couch
dataStoreType=es

#mongo-db related properties
mongo.host=localhost
mongo.useAuth=false
mongo.user=sathis
mongo.password=mongo
mongo.db=proliphiq
mongo.collection=users

#couch-db related settings
couch.host=localhost
couch.db=test
couch.port=5984

#Elastic-Search related properties
es.cluster.name=elasticsearch-test
#if you want to connect to multiple nodes for parallel indexing
# give them as comma separated.
es.hosts=localhost
es.index.name=ojdbc
es.index.type=custom_index_1

data-config.xml:
<dataConfig>
        <dataSource driver="oracle.jdbc.OracleDriver" url="jdbc:oracle:thin:@...." user="x" password="y" batchSize="500" maxRows="500000"/>
                <document name="doc">
                        <entity name="q1" query="select notice_id, country, indexing_state from ep_notices">
                                <field column="notice_id" name="notice_id" type="int"/>
                                <field column="country" name="notice_country" type="string"/>
                                <field column="indexing_state" name="notice_indexing_state" type="int"/>
                                <entity name="q2" query="select text_type, lang, text from ep_notice_text where text_type in ('summary', 'title', 'full_text', 'eu_tx', 'eu_ot') and notice_id = '${q1.notice_id}'">
                                        <field column="text_type" name="notice_text_type" type="string"/>
                                        <field column="lang" name="notice_lang" type="string"/>
                                        <field column="text" name="notice_text" type="string" multiValued="true"/>
                                </entity>
                        </entity>
                </document>
</dataConfig>


When I run the shell script I receive the following exception:
Jan 10, 2013 9:08:04 AM org.elasticsearch.plugins
INFO: [Zero] loaded [], sites []
Jan 10, 2013 9:08:06 AM net.sathis.export.sql.DataImporter loadDataConfig
INFO: Data Configuration loaded successfully
Jan 10, 2013 9:08:06 AM net.sathis.export.sql.DataImporter doFullImport
SEVERE: *****  Data import failed. **********
 Reason is :
java.lang.NumberFormatException: null
    at java.lang.Integer.parseInt(Integer.java:454)
    at java.lang.Integer.valueOf(Integer.java:582)
    at net.sathis.export.sql.DocBuilder.execute(DocBuilder.java:101)
    at net.sathis.export.sql.DataImporter.doFullImport(DataImporter.java:174)
    at net.sathis.export.sql.DataImporter.doDataImport(DataImporter.java:93)
    at net.sathis.export.sql.SQLToNoSQLImporter.main(SQLToNoSQLImporter.java:19)

[sathis]

Sorry for late response. Looks like your table has null values for integer field types. Either make them zero or modify the code (if you wish, i can help) to allow null values.

Regards

 Sathis