CKAN java library

[Skaros Ilias]

Hi all,
I am trying to create a java application that would eventually get use of the API and mainly upload files(csv and zip) to a ckan installation(currently a local installation under a VM).
The problem is that i cant find any libraries that would help me do this. I tried to do it manually, using json and http commands but i hit a wall (more than once)
I have found some that claim to be java libs, but either i cant get them to work, or they are not what they say they are.

Mostly i think ckan clients are written in Python, the problem is that i dont know python. I would really love if there was a library that i could just use and do all the hard work for me.

In case someone might be able to help this is the code I did in java

static String myApiKey="fa0499d1-ffda-4590-82b3-4afdb9c91576"; 
static String uploadFileName="/home/ilias/2013/05/csv/2013_05_15.csv";
 public static void uploadFile()

 { HttpClient httpclient = new DefaultHttpClient();

     Date now=new Date();
     File file = new File(uploadFileName);
     httpclient = new DefaultHttpClient();
     try {

         SimpleDateFormat dateFormatGmt = new  SimpleDateFormat("yyyyMMMddHHmmss");
         dateFormatGmt.setTimeZone(TimeZone.getTimeZone("GMT"));
         String date=dateFormatGmt.format(new Date());
   /*      First we specify the data             */
          HttpEntity reqEntity = MultipartEntityBuilder.create()
            .addTextBody("package_id","test2")
            .addTextBody("url",page)
            .setCharset(Charset.forName("utf-8"))
            .addTextBody("file", date +"/"+uploadFileName)              
            .build();
  /*    Set up the headers       */
        HttpPost postRequest = new HttpPost(page+"/api/action/resource_create/");
        postRequest.setEntity(reqEntity);      
        postRequest.setHeader("X-CKAN-API-Key", myApiKey);
 /*     execute       */

        System.out.println("request2: "+postRequest.toString()+"\n"+postRequest.getRequestLine().toString());

        Header[]a=postRequest.getAllHeaders();
        HttpResponse response = httpclient.execute(postRequest);
/*     get respond    */
       int statusCode = response.getStatusLine().getStatusCode();
        BufferedReader br = new BufferedReader(
        new InputStreamReader((response.getEntity().getContent())));

        String line;
        while ((line = br.readLine()) != null) {
                System.out.println("__"+line);
}
if(statusCode!=200){
   System.out.println("statusCode ==" +statusCode);
}
}catch (IOException ioe) {
System.out.println(ioe);
 } finally {
httpclient.getConnectionManager().shutdown();
}

/*Upload the file */
  file = new File(uploadFileName);
httpclient = new DefaultHttpClient();
try {
       FileBody bin = new FileBody(file,ContentType.TEXT_PLAIN,now.toString()+file.getName());

/*   add the data     */
       HttpEntity reqEntity = MultipartEntityBuilder.create()
        .addPart("file", bin)
        .addTextBody("key", now.toString()+file.getName())
        .build();

       HttpPost postRequest = new HttpPost(page+"/api/action/resource_create/");
       postRequest.setEntity(reqEntity);
       postRequest.setHeader("X-CKAN-API-Key", myApiKey);
       HttpResponse response = httpclient.execute(postRequest);
       int statusCode = response.getStatusLine().getStatusCode();
       BufferedReader br = new BufferedReader(
               new InputStreamReader((response.getEntity().getContent())));

       String line;
       while ((line = br.readLine()) != null) {
         System.out.println("+"+line);
       }
       if(statusCode!=200){
          System.out.println("statusCode =!=" +statusCode);
       }
}catch (IOException ioe) {
System.out.println(ioe);
} finally {
httpclient.getConnectionManager().shutdown();
}
}


And the error i was getting is a 301 Error

I also tried this, using httpGet
URIBuilder builder = new URIBuilder();
        builder.setScheme("http").setHost("192.168.1.1:5000").setPath("/api/action/resource_create/")
         .setCharset(Charset.forName("utf-8"))
            .setParameter("package_id", "test2")
            .setParameter("url", "http://www.example.com");

URI uri = builder.build();
HttpGet postRequest = new HttpGet(uri);
postRequest.setHeader("X-CKAN-API-Key", myApiKey);

Which returned a more understandable error of

"Bad request - JSON Error: No request body data"

Any help/guideline is appreciated

[Matthew Fullerton]

ckan-dev in cc.

Hmm, it might be simpler to just write a Java wrapper around https://github.com/ckan/ckanapi using Jython (http://www.jython.org/)

However, if you want to start from scratch, I can maybe give you a little help to get started; this is how we access the API for the few things we need from inside Java (url is the API endpoint, you can include url arguments; apiKey is your API key, needed if the dataset is private):

package package.name;

import javax.ws.rs.client.Client;

import javax.ws.rs.client.ClientBuilder;

import javax.ws.rs.client.WebTarget;

import javax.ws.rs.core.MediaType;

import javax.ws.rs.core.Response;

import org.json.JSONException;

import org.json.JSONObject;

public class CkanApiClient {

    private JSONObject getJsonResultForResource(String url, String apiKey) throws JSONException {

        JSONObject resourceProperties = null;

        Client client = ClientBuilder.newClient();

        WebTarget resourceTarget = client.target(url);

        //Set API key

        Response response = resourceTarget.request(MediaType.APPLICATION_JSON).header("X-CKAN-API-Key", apiKey).get();

        String responseString = response.readEntity(String.class);

        resourceProperties = new JSONObject(responseString);

        return resourceProperties;

    }

}

What's not in there is sending a JSON body which you will need for e.g. creating datasets.

Best,

Matt

--

Matthew Fullerton

Freelance Software Developer und EXIST Stipend holder with the start up project "Tapestry" - http://www.smartlane.de/

--
You received this message because you are subscribed to the Google Groups "CKAN Global User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ckan-global-user-...@googlegroups.com.
To post to this group, send email to ckan-global...@googlegroups.com.
Visit this group at http://groups.google.com/group/ckan-global-user-group.
To view this discussion on the web, visit https://groups.google.com/d/msgid/ckan-global-user-group/921a86fd-38e1-4701-97d7-8ae1256fb7b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matthew Fullerton]

Me again. To answer your more specific issue with the Bad Request error: you need to send what you are sending as get parameters as keys within a JSON body. That is a POST request where the body is in JSON. You can play with the API using an extension like Postman in Chrome where you can set body type JSON and use the API docs as guidance. Whether httpget has a way to make post requests with JSON bodies, I don't know, but the code I sent can do it (append .post after the .header segment in the code: http://docs.oracle.com/javaee/7/api/javax/ws/rs/client/SyncInvoker.html#post(javax.ws.rs.client.Entity))

--

Matthew Fullerton

Freelance Software Developer und EXIST Stipend holder with the start up project "Tapestry" - http://www.smartlane.de/

[Skaros Ilias]

Hi Matthew, Thanks for your reply.
As a matter of fact i did try to use the python client using jython, but didnt work out for me. I finally turned to ckan4j, which is a java client, but my problem is that they use maven, and i am not familiar with it. I have some trouble setting it up, but it seems to be the best solution for me right now

To unsubscribe from this group and stop receiving emails from it, send an email to ckan-global-user-group+unsub...@googlegroups.com.

[Matthew Fullerton]

My advice with Maven is always to try Netbeans first; its quite good at just opening Maven projects as projects and figuring everything out.

To unsubscribe from this group and stop receiving emails from it, send an email to ckan-global-user-...@googlegroups.com.

To post to this group, send email to ckan-global...@googlegroups.com.
Visit this group at http://groups.google.com/group/ckan-global-user-group.
To view this discussion on the web, visit https://groups.google.com/d/msgid/ckan-global-user-group/921a86fd-38e1-4701-97d7-8ae1256fb7b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "CKAN Global User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ckan-global-user-...@googlegroups.com.

To post to this group, send email to ckan-global...@googlegroups.com.
Visit this group at http://groups.google.com/group/ckan-global-user-group.

To view this discussion on the web, visit https://groups.google.com/d/msgid/ckan-global-user-group/50232711-f7a4-4b81-975d-d9dcbafbedda%40googlegroups.com.

[Skaros Ilias]

Netbeans is alot heavier than eclipse for my pc as it is right now. I think i managed to get it to work, I just had to do some manual imports. The problem is that i dont know how to use the commands of Ckan5j. I contacted the author and waiting for some javaDocs, or some info anyway. Thanks again

To unsubscribe from this group and stop receiving emails from it, send an email to ckan-global-user-group+unsub...@googlegroups.com.

To post to this group, send email to ckan-global...@googlegroups.com.
Visit this group at http://groups.google.com/group/ckan-global-user-group.
To view this discussion on the web, visit https://groups.google.com/d/msgid/ckan-global-user-group/921a86fd-38e1-4701-97d7-8ae1256fb7b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "CKAN Global User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ckan-global-user-group+unsub...@googlegroups.com.

To post to this group, send email to ckan-global...@googlegroups.com.
Visit this group at http://groups.google.com/group/ckan-global-user-group.