Able to import only 20 events using batch import

17 views
Skip to first unread message

Yasho R

unread,
Feb 4, 2016, 8:12:42 AM2/4/16
to predictionio-dev


Hello Team,

i'm using Ubuntu OS, installed PredictionIO with Elasticsearch/HBase backend storage.
Used Similarity Template to start predicting similar movie results.
i have imported events successfully, but the problem in my case i was trying to import attached events.json file. my events.json file having 100 events. but i'm able to import only 20 events for each app in the engine.

here is my successfully imported events command history,

pio import --appid 3 --input /home/administrator/imdbpy/events.json
[INFO] [Runner$] Submission command: /home/administrator/PredictionIOES2/vendors/spark-1.5.1/bin/spark-submit --class io.prediction.tools.imprt.FileToEvents --files file:/home/administrator/PredictionIOES2/conf/log4j.properties,file:/home/administrator/PredictionIOES2/vendors/hbase-1.0.0/conf/hbase-site.xml --driver-class-path /home/administrator/PredictionIOES2/conf:/home/administrator/PredictionIOES2/vendors/hbase-1.0.0/conf file:/home/administrator/PredictionIOES2/lib/pio-assembly-0.9.5.jar --appid 3 --input file:/home/administrator/imdbpy/events.json --env PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/home/administrator/.pio_store,PIO_STORAGE_SOURCES_HBASE_HOME=/home/administrator/PredictionIOES2/vendors/hbase-1.0.0,PIO_HOME=/home/administrator/PredictionIOES2,PIO_FS_ENGINESDIR=/home/administrator/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/home/administrator/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/home/administrator/PredictionIOES2/vendors/elasticsearch-1.4.4,PIO_FS_TMPDIR=/home/administrator/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/home/administrator/PredictionIOES2/conf,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
[WARN] [Utils] Your hostname, vs-yrubuntu resolves to a loopback address: 127.0.1.1; using 10.10.100.179 instead (on interface enp0s3)
[WARN] [Utils] Set SPARK_LOCAL_IP if you need to bind to another address
[INFO] [Remoting] Starting remoting
[INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp://spark...@10.10.100.179:33498]
[WARN] [MetricsSystem] Using default name DAGScheduler for source because spark.app.id is not set.
[INFO] [FileToEvents$] Events are imported.
[INFO] [FileToEvents$] Done.


and here is screenshot for my imported events list in web browser,


Kindly advise me in this.

its kind of an urgency.


Thanks in Advance,

Yasho

events.json

Kenneth Chan

unread,
Feb 4, 2016, 2:02:37 PM2/4/16
to Yasho R, predictionio-dev
Reply all
Reply to author
Forward
0 new messages