error in using sentiment api with R

[Giorgio Spedicato]

I am using R + RCCurl + Dandelion sentiment API extractor to perform some sentiment analysis.

I am using this code

dataTxtSent <- function(input, app_id, app_key,  language='auto') {

  nex <- 'https://api.dandelion.eu/datatxt/sent/v1'

  params <- list()

  params['$app_id'] <- app_id

  params['$app_key'] <- app_key

  if (grepl("^https?://", input)){

    params['url'] <- input

  } else {

    params['text'] <- input

  }

  params['lang'] <- language

  cert <- system.file("CurlSSL/cacert.pem", package = "RCurl") 

  appReply <- fromJSON(getForm(nex, .params=params, .checkParams = TRUE, .opts = mycurlopts))

  out<-data.frame(type=appReply$sentiment$type,score=appReply$sentiment$score)

  return(out)

}

app_id='71358875'

app_key='342f452f013b19e5d02d699d235a52d4'

results <- dataTxtSent(input='Oggi il tempo fa schifo', app_id, app_key, language = 'auto')

but I receive this error : " Error: Argument 'txt' must be a JSON string, URL or file. "

How can I fix it?

Also I was suggested to use Connection = "keep-alive"

Since I use a proxy, my curl options are below exposed

mycurlopts <- list(

  proxy         = "myproxy", 

  proxyusername = "myusername", 

  proxypassword = "mypassword", 

  proxyport     = 80,

#  Connection = "keep-alive",

  ssl.verifypeer = FALSE

)

The question is: where have I to specify the connection.

Thanks in advance for your time.

[sam...@spaziodati.eu]

I am having hard time replicating the Error you pasted. The same code works for me (using proxies as well).
Anyway I looked at jsonlite libray: https://github.com/jeroenooms/jsonlite/blob/master/R/fromJSON.R
That error is returned whenever the argument passed to fromJSON is neither a string nor a file.
(Here the code throwing the exception
# check type
if (!is.character(txt) && !is(txt, "connection")) {
stop("Argument 'txt' must be a JSON string, URL or file.")
}
)
I guess, then, that for some reason the result of the request getForm(nex, .params=params, .checkParams = TRUE, .opts = mycurlopts) does not fulfill such conditions. Could you print it and share?

[Giorgio Spedicato]

The full code is below:

library("jsonlite")

library("RCurl")

library(XML)

#options(RCurlOptions = list(proxy = "proxybo.servizi.gr-u.it", proxyport = 80))

mycurlopts <- list(

  proxy         = "myoproxy", 

  proxyusername = "myun", 

  proxypassword = "mypw", 

  proxyport     = 80,

#  Connection = "keep-alive",

  ssl.verifypeer = FALSE

)

dataTxtNex <- function(input, app_id, app_key, include, min_confidence=.6, language='auto', social=F) {

  nex <- 'https://api.dandelion.eu/datatxt/nex/v1/'

  params <- list()

  params['$app_id'] <- app_id

  params['$app_key'] <- app_key

  if (grepl("^https?://", input)){

    params['url'] <- input

  } else {

    params['text'] <- input

  }

  params['min_confidence'] <- min_confidence

  params['social'] <- social

  params['include'] <- include

  params['lang'] <- language

  cert <- system.file("CurlSSL/cacert.pem", package = "RCurl") 

  dataTxtReply <- fromJSON(getForm(nex, .params=params, .checkParams = TRUE, cainfo = cert))

  annotations <- dataTxtReply$annotations

  if (length(annotations) > 1) {

    return(annotations)

  } else {

    return('No entities found')

  }

}

dataTxtSent <- function(input, app_id, app_key,  language='auto') {

  nex <- 'https://api.dandelion.eu/datatxt/sent/v1'

  params <- list()

  params['$app_id'] <- app_id

  params['$app_key'] <- app_key

  if (grepl("^https?://", input)){

    params['url'] <- input

  } else {

    params['text'] <- input

  }

  params['lang'] <- language

  cert <- system.file("CurlSSL/cacert.pem", package = "RCurl") 

  appReply <- fromJSON(getForm(nex, .params=params, .checkParams = TRUE, .opts = mycurlopts))

  out<-data.frame(type=appReply$sentiment$type,score=appReply$sentiment$score)

  return(out)

}

app_id='71358875'

app_key='342f452f013b19e5d02d699d235a52d4'

results <- dataTxtSent(input='Oggi il tempo fa schifo', app_id, app_key, language = 'auto')

The error below

 Error: Argument 'txt' must be a JSON string, URL or file. 

[sam...@spaziodati.eu]

What I need in order to understand what's going wrong is the value of

getForm(nex, .params=params, .checkParams = TRUE, .opts = mycurlopts)

That is: do not pass it to fromJSON (which is causing the error) and simply return its value. Then we'll try to figure out the problem.

[Giorgio Spedicato]

I received this code from a representative of your company (see my previous post).

So, I reformulate the question:

IS IT POSSIBLE TO HAVE AN EXAMPLE OF CALLING THE SENTIMENT ANALYSIS API BY R?

Best

[sam...@spaziodati.eu]

There is some sort of misunderstanding, I dare say.

In the R code making requests to our API there was no usage of proxies which is likely to be the source of errors.

Anyway, rather than keep conjecturing we may understand the problem by simply reading the reply to the requests since the problem is in "parsing" the value of getForm(nex, .params=params, .checkParams = TRUE, .opts = mycurlopts) which is not a valid Json.

So I kindly ask to modify dataTxtSent in the following way:

dataTxtSent <- function(input, app_id, app_key, language='auto') {
nex <- 'https://api.dandelion.eu/datatxt/sent/v1'
params <- list()
params['$app_id'] <- app_id
params['$app_key'] <- app_key
if (grepl("^https?://", input)){
params['url'] <- input
} else {
params['text'] <- input
}
params['lang'] <- language
cert <- system.file("CurlSSL/cacert.pem", package = "RCurl")

appReply <- getForm(nex, .params=params, .checkParams = TRUE, .opts = mycurlopts)
return(appReply)
}

then let it run:

results <- dataTxtSent(input='Oggi il tempo fa schifo', app_id, app_key, language = 'auto')

and then share with us the value of "result".

Kind regards,
Nicola

[Giorgio Spedicato]

It appears to return a value but it is not understandable.....

app_id='71358875'

app_key='342f452f013b19e5d02d699d235a52d4'

  

results <- dataTxtSent(input='Troppo traffico alle 18, che palle!', app_id, app_key, language = 'auto')

gives

results

  [1] 7b 22 74 69 6d 65 22 3a 30 2c 22 73 65 6e 74 69 6d 65 6e 74 22 3a 7b 22 74 79 70 65

 [29] 22 3a 22 6e 65 67 61 74 69 76 65 22 2c 22 73 63 6f 72 65 22 3a 2d 30 2e 38 7d 2c 22

 [57] 6c 61 6e 67 22 3a 22 69 74 22 2c 22 6c 61 6e 67 43 6f 6e 66 69 64 65 6e 63 65 22 3a

 [85] 31 2e 30 2c 22 74 69 6d 65 73 74 61 6d 70 22 3a 22 32 30 31 35 2d 30 37 2d 32 38 54

[113] 31 34 3a 30 36 3a 35 37 2e 35 39 31 22 7d

[sam...@spaziodati.eu]

Thanks. There is some problem in encoding support on your side.

You can see that by using any convert from hexadecimal to string: for example you can copy and paste the value of the result (removing the line marks) in http://software.hixie.ch/utilities/cgi/unicode-decoder/utf8-decoder and select Hexadecimal from the drop-down menu: you'll then read:

As raw characters:
{"time":0,"sentiment":{"type":"negative","score":-0.8},"lang":"it","langConfidence":1.0,"timestamp":"2015-07-28T14:06:57.591"}

May I know which version for R and RCurl you're currently using? (Mine are: 3.1.3 and RCurl 1.95-4.5).

[Giorgio Spedicato]

On another PC (whitout the proxy) it appears to work but after some request it returns to me

 Error: Bad Request 

shall I change the  curl option allowing Keep Alive? How can I do it?

mycurlopts2 <- list(

  ssl.verifypeer = FALSE

)

Thank you very much for the support and patience