Ingesting Data from "CDAP stream client for Java"

[Young Liu]

Hi Folks,

I want to pass data to stream through the "stream client".

Here is my code:

————————————————————————————————————

package com.hackspark.rtt;

import java.io.BufferedReader;

import java.io.File;

import java.io.FileReader;

import java.io.IOException;

import org.apache.commons.io.Charsets;

import co.cask.cdap.client.StreamClient;

import co.cask.cdap.client.StreamWriter;

import co.cask.cdap.client.rest.RestStreamClient;

public class StreamCtrl{

    public static void main(String... args) throws IOException{

        StreamClient streamClient = new RestStreamClient.Builder("localhost",10000).build();

        streamClient.create("gps");

        StreamWriter streamWriter= streamClient.createWriter("gps");

        File file = null;

        try{

            file = new File("/Documents/map.txt");

        }catch(NullPointerException e){

            //ignore

        }

        BufferedReader reader = null;

        String line = null;

        try{

            reader = new BufferedReader(new FileReader(file));

            while( (line = reader.readLine()) != null ){

                streamWriter.write(line,Charsets.UTF_8);

            }

        }catch (IOException e){

            e.printStackTrace();

        }catch (NullPointerException ee){

            ee.printStackTrace();

        }finally{

            reader.close();

        }

    }

}

——————————————————————————————————————————

I used the maven generate this project , and i had added this dependency following the guide from http://docs.cask.co/cdap/current/en/developers-manual/ingesting-tools/cdap-stream-clients-java.html

And I can successfully "mvn clean package" and then generate the .jar document.After this , I set the main-class enter point.

But,I encoutered this error:

——————————————————————————————————————————

java -jar streamclient-1.0-SNAPSHOT.jar

Exception in thread "main" java.lang.NoClassDefFoundError: co/cask/cdap/client/StreamClient

at java.lang.Class.getDeclaredMethods0(Native Method)

at java.lang.Class.privateGetDeclaredMethods(Class.java:2615)

at java.lang.Class.getMethod0(Class.java:2856)

at java.lang.Class.getMethod(Class.java:1668)

at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)

at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)

Caused by: java.lang.ClassNotFoundException: co.cask.cdap.client.StreamClient

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

... 6 more

————————————————————————————————————————————

The solutions I had tried:

1.inspect the MANIFEST.MF :

there are these entries:

--Embed-Directory: lib

--Embedded-Artifacts: there has "lib/cdap-stream-client-1.0.1.jar" in this entry.

--Bundle-ClassPath:there has "lib/cdap-stream-client-1.0.1.jar" in this entry

--Main-Class:

and so on.

IS there something wrong above?

2.I had downloaded the cdap-stream-client.jar from http://mvnrepository.com/artifact/co.cask.cdap/cdap-stream-client/1.0.1.

And I add this to the classpath , BUT there is a new error:

————————————————————————————————————————————————————

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/conn/HttpClientConnectionManager

at co.cask.cdap.client.rest.RestStreamClient$Builder.build(RestStreamClient.java:224)

at cn.siat.hackspark.StreamCtrl.main(StreamCtrl.java:17)

Caused by: java.lang.ClassNotFoundException: org.apache.http.conn.HttpClientConnectionManager

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

... 2 more

————————————————————————————————————————————

It seems like the dependency is not right. Am i right?

How can i figure out the solution?

Cheers,

-Young

[Nitin Motgi]

Hi Young,

Could you please send us the POM file that u have. It would help debugging the issue you are seeing.

Thanks,

Nitin

###

Random auto-corrects and typos are my special gift to you. When I forward they are from others. 

--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/11e46413-7d49-4423-8907-82be6bac9296%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shankar Selvam]

Hi Young, 

How are you building your executable jar, did you use any plugin to create the executable jar ?

I tried building the jar with the shade plugin and it works fine. 

http://maven.apache.org/plugins/maven-shade-plugin/examples/executable-jar.html 

If you are using other approaches, then as per Nitin's suggestion, if you can share the pom.xml,  it will help debug the issue.

To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/EA176628-686D-460B-A3A8-473651EDDEBD%40cask.co.

[Young Liu]

Hi shankar,

Sorry for slow to respond(for three days out of school), I appreciate for your advise .

After referring to your method , i adopted the shade plugin too, and it works fine too.

Thank u again. I am not familiar with maven so that your advice is important to me. :D.

Cheers,

-Young
在 2014年12月11日星期四UTC+8下午3时31分10秒，shankar写道：

[Young Liu]

Hi Nitin,

Sorry for slow to reponse(for three days out of school), I had referred the shankar's method, and it works fine.

Thank u for your enthusiastic help! 

Cheers,

-Young

在 2014年12月11日星期四UTC+8下午2时56分11秒，Nitin Motgi写道：