data ingestion aletrnatively to drah&drop

[Pasquale Iodice]

Hi all,
Could someone explain to me how to load a file (.pdf .doc .xml .png) in my workspace alternatively to drag&drop method? Thanks!

[David Singley]

The Import tool accepts a option to place imported data into a specified workspace ID, see https://github.com/lumifyio/lumify/blob/master/tools/import/src/main/java/io/lumify/tools/Import.java#L60

[Leo nardo Furio]

Just to be more clear!

Herewith the script to use the import file 

classpath=$(/home/lumify/lumify/bin/classpath.sh tools/import)

java \

-Xmx1024M \

-Djava.awt.headless=true \

-Dfile.encoding=UTF-8 \

-Djava.library.path=$LD_LIBRARY_PATH \

-classpath ${classpath} \

io.lumify.tools.Import \

--datadir=/home/lumify/importFiles/ \ 

--queuedups=true\

--visibilitysource=""\

--workspaceid=WORKSPACE_d7bef7e34d364f34b332a9efd7035afd

Obviously change the input data with your data!

Leo

[Pasquale Iodice]

Hi Leo, thanks for your support

I made a new script as you suggested suitably modifying "datadir" and "workpaceid".

This is my script:

#!/bin/bash

classpath=$(/home/pasquale/lumify/bin/classpath.sh tools/import)

java \

-Xmx1024M \

-Djava.awt.headless=true \

-Dfile.encoding=UTF-8 \

-Djava.library.path=$LD_LIBRARY_PATH \

-classpath ${classpath} \

io.lumify.tools.Import \

--datadir=/home/pasquale/lumify/importFiles/ \ 

--queuedups=true\

--visibilitysource=""\

--workspaceid=WORKSPACE_494b3c05a371424f947520ca3b154ce1

And this is my output error

pasquale@ubuntu:~/lumify/bin$ ./scriptImport.sh 

running maven to calculate the classpath...

Error: Could not find or load main class Scanning

./scriptImport.sh: line 15: --queuedups=true--visibilitysource=a--workspaceid=WORKSPACE_494b3c05a371424f947520ca3b154ce1: command not found

[Leo nardo Furio]

Have you compiled tools/import before run the script ?

mvn package -pl ./tools/import  -am -DskipTests

Leo

[Pasquale Iodice]

I just compiled it

 Reactor Summary:

[INFO] 

[INFO] Lumify Root: Build Tools ........................... SUCCESS [ 32.114 s]

[INFO] Lumify ............................................. SUCCESS [ 18.339 s]

[INFO] Lumify: Web ........................................ SUCCESS [  0.029 s]

[INFO] Lumify: Web: Client API ............................ SUCCESS [03:50 min]

[INFO] Lumify: Core ....................................... SUCCESS [  0.050 s]

[INFO] Lumify: Core ....................................... SUCCESS [11:30 min]

[INFO] Lumify: Core: Plugins .............................. SUCCESS [  0.077 s]

[INFO] Lumify: Core: Plugin: Model: BigTable .............. SUCCESS [ 45.784 s]

[INFO] Lumify: Core: Plugin: Model: RabbitMQ .............. SUCCESS [  7.292 s]

[INFO] Lumify: Core: Plugin: Model: Secure Graph .......... SUCCESS [02:22 min]

[INFO] Lumify: Tools ...................................... SUCCESS [  0.690 s]

[INFO] Lumify: Tools: Import .............................. SUCCESS [  0.440 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

but I have the same error: 

bash-4.1# ./scriptImport.sh

./scriptImport.sh: line 3: /home/pasquale/lumify/bin/classpath.sh: No such file or directory

Unrecognized option: --datadir=/home/pasquale/lumify/importFiles/

Error: Could not create the Java Virtual Machine.

Error: A fatal exception has occurred. Program will exit.

./scriptImport.sh: line 15: --queuedups=true--visibilitysource=--workspaceid=WORKSPACE_494b3c05a371424f947520ca3b154ce1: command not found

[Leo nardo Furio]

The problem seems to be caused from the location of classpath file...

: /home/pasquale/lumify/bin/classpath.sh: No such file or directory

Leo

[Pasquale Iodice]

It was a syntax error at last line: no symbol "\" at the end. Now if I try to run it I have this output: 

running maven to calculate the classpath...

Error: Could not find or load main class Scanning

and if I try to run it in sudo mode I have this output:

not running maven, using cached classpath.

Exception in thread "main" java.lang.NoClassDefFoundError: io/lumify/core/cmdline/CommandLineBase

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:800)

at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)

at java.net.URLClassLoader.access$100(URLClassLoader.java:71)

at java.net.URLClassLoader$1.run(URLClassLoader.java:361)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)

Caused by: java.lang.ClassNotFoundException: io.lumify.core.cmdline.CommandLineBase

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

... 13 more