debugging a knitr bad request error

[Stednick, Zach]

Hi

 

I am trying to use knitr on knitr-examples/009-slides.Rmd from github and I keep getting this error:

Error in process_file(text, output) : 

  Quitting from lines 29-31: (009-slides.Rmd) Error : Bad Request

 

Calls: knit -> process_file

Execution halted

knitr terminated with status 1

 

Those lines correspond to a ggplot2 plot code chunk which works fine if I use it outside knitr.

 

I am running RStudio 0.96.331 on Windows and ggplot2 is installed with all the other packages (and loads fine when I run library(ggplot2))

 

> find.package('ggplot2')

[1] "C:/Program Files/R/R-2.15.1/library/ggplot2"

> packageVersion('knitr')

[1] ‘0.8’

> packageVersion('ggplot2')

[1] ‘0.9.2.1’

 

Any ideas what to try next?  I did just switch from Windows XP to Win 7 and started noticing this problem then, but am not sure that this is related.

 

Zach

[Yihui Xie]

Bad request is an indication of a web problem. I believe that is a
problem of Imgur, since I used the imgur_upload() function in the Rmd
document so that the plot can show up on Github:
https://github.com/yihui/knitr-examples/blob/master/009-slides.md

Remove this line and it should work: opts_knit$set(upload.fun = imgur_upload)

Regards,
Yihui
--
Yihui Xie <xiey...@gmail.com>
Phone: 515-294-2465 Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA

[Yihui Xie]

OK, I can reproduce the problem now. I'll see what is wrong there. Thanks!

Regards,
Yihui
--
Yihui Xie <xiey...@gmail.com>
Phone: 515-294-2465 Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA

On Mon, Oct 22, 2012 at 3:23 PM, Aaron Wolen <thea...@gmail.com> wrote:
> I'm not sure this is a web problem.
>
> I'm getting the same bad request response when opts_knit$set(upload.fun =
> imgur_upload) but have no issue uploading images using imguRupload (from the
> imguR package) with the same api key.
>
>

[Yihui Xie]

It looks like Duncan borrowed a function from Hadley on Sept 29 which
broke the imgur_upload() function in knitr:
https://github.com/omegahat/RCurl/commit/8cb23e7e0704fd1015bc3df83bafd31304c0d7c3
Version 1.91-1 works fine:

wget http://www.omegahat.org/R/src/contrib/RCurl_1.91-1.tar.gz
R CMD INSTALL RCurl_1.91-1.tar.gz

To Duncan: could you take a look at knitr::imgur_upload and see what
is wrong with RCurl::postForm?
https://github.com/yihui/knitr/blob/master/R/utils-upload.R Thanks!

Regards,
Yihui
--
Yihui Xie <xiey...@gmail.com>
Phone: 515-294-2465 Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA

On Mon, Oct 22, 2012 at 3:23 PM, Aaron Wolen <thea...@gmail.com> wrote:
> I'm not sure this is a web problem.
>
> I'm getting the same bad request response when opts_knit$set(upload.fun =
> imgur_upload) but have no issue uploading images using imguRupload (from the
> imguR package) with the same api key.
>
>

[Yihui Xie]

I did spend an hour trying to trace down what was going on before
turning to you for help; since I do not really know much about the
RCurl internals, what I could do at best was to try different
revisions and versions of RCurl. I ended up with the guess in my last
email because RCurl worked until that commit.

Sorry if there is misunderstanding. I'll give another try to look at
that commit.

Regards,
Yihui
--
Yihui Xie <xiey...@gmail.com>
Phone: 515-294-2465 Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA

On Tue, Oct 23, 2012 at 12:51 AM, Duncan Temple Lang
<dtemp...@ucdavis.edu> wrote:

>
>
> On 10/22/12 8:50 PM, Yihui Xie wrote:
>> It looks like Duncan borrowed a function from Hadley on Sept 29 which
>> broke the imgur_upload() function in knitr:
>

> I love when people proclaim their own version of reality.
> Why not start over and think what actually happened

>
>> https://github.com/omegahat/RCurl/commit/8cb23e7e0704fd1015bc3df83bafd31304c0d7c3
>> Version 1.91-1 works fine:
>>
>> wget http://www.omegahat.org/R/src/contrib/RCurl_1.91-1.tar.gz
>> R CMD INSTALL RCurl_1.91-1.tar.gz
>>
>> To Duncan: could you take a look at knitr::imgur_upload and see what
>> is wrong with RCurl::postForm?
>> https://github.com/yihui/knitr/blob/master/R/utils-upload.R Thanks!
>

> Perhaps rather than asking me to do work, you might suggest changes
> based on what Hadley wanted and affected your derivative work.
>
> D.
>

[Aaron Wolen]

Thanks for looking into this, Yihui. Creating html reports with knitr/Rmarkdown has become such an integral part of my workflow that it's very disruptive when something breaks down. 

For others out there who are running into this issue, I used the imguR package to implement the following workaround:

opts_knit$set(upload.fun = function(file, key = "60e9e47cff8483c6dc289a1cd674b40f") {

  res <- imguR::imguRupload(file, apikey = key)

  structure(res[["links.original"]], XML = res)

})

[Yihui Xie]

I spent another three hours digging into this problem, and I'm
relatively confident that this commit broke the upload function:

https://github.com/omegahat/RCurl/commit/b1f88c14e85893db10df8b264bc3ea2ec8085072#L8L120

It is unclear to me why .params was changed to lapply(.params, as,
"character") instead of as.list(.params), since the commit message
"writing chapter" was vague, but the consequence is that it breaks the
results from fileUpload(), e.g.

> library(RCurl)
> .params = list(file = fileUpload(file.path(R.home(), 'NEWS')))
> lapply(.params, as, "character")
$file
[1] "/usr/lib/R/NEWS" "character(0)" "character(0)"

> # it should really be this:
> .params
$file
$filename
[1] "/usr/lib/R/NEWS"

$contents
character(0)

$contentType
character(0)

attr(,"class")
[1] "FileUploadInfo"


It I revert that single change, postForm() will be able to upload
files again. However, even if I do that, there is still a problem
which I cannot figure out: the file seems to be uploaded twice. Here
is one of the results I got when I tried to upload an image to imgur
with knitr::imgur_upload():

> str(res)
atomic [1:1] <?xml version="1.0" encoding="utf-8"?>
<upload><image><name>~/tmp/sweave/figure/lmex.png</name><title/><caption/><hash>vaSmW</hash><deletehash>9PbLOHIIKPczb3h</deletehash><datetime>2012-10-24
02:29:25</datetime><type>image/png</type><animated>false</animated><width>504</width><height>504</height><size>5869</size><views>0</views><bandwidth>0</bandwidth></image><links><original>http://i.imgur.com/vaSmW.png</original><imgur_page>http://imgur.com/vaSmW</imgur_page><delete_page>http://imgur.com/delete/9PbLOHIIKPczb3h</delete_page><small_square>http://i.imgur.com/vaSmWs.jpg</small_square><large_thumbnail>http://i.imgur.com/vaSmWl.jpg</large_thumbnail></links></upload>
HTTP/1.1 100 Continue

HTTP/1.1 200 Ok
Server: nginx
Date: Wed, 24 Oct 2012 02:29:26 GMT
Content-Type: application/xml
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: IMGURSESSION=ll6v92n7gl71pg0v9i8ffkfde3; path=/; domain=.imgur.com
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Set-Cookie: _nc=1; path=/; domain=.imgur.com; httponly
Access-Control-Allow-Origin: *
X-RateLimit-Limit: 500
X-RateLimit-Remaining: 410
X-RateLimit-Reset: 1351047952
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0

<?xml version="1.0" encoding="utf-8"?>
<upload><image><name>~/tmp/sweave/figure/lmex.png</name><title/><caption/><hash>pMuuJ</hash><deletehash>O1QBH9SRJQ72oWO</deletehash><datetime>2012-10-24
02:29:26</datetime><type>image/png</type><animated>false</animated><width>504</width><height>504</height><size>5869</size><views>0</views><bandwidth>0</bandwidth></image><links><original>http://i.imgur.com/pMuuJ.png</original><imgur_page>http://imgur.com/pMuuJ</imgur_page><delete_page>http://imgur.com/delete/O1QBH9SRJQ72oWO</delete_page><small_square>http://i.imgur.com/pMuuJs.jpg</small_square><large_thumbnail>http://i.imgur.com/pMuuJl.jpg</large_thumbnail></links></upload>

- attr(*, "Content-Type")= Named chr "application/xml"
..- attr(*, "names")= chr ""


----

The imguR package was apparently broken as well; its author gave up
postForm() and turned to Hadley's httr package for the POST() function
instead, which uses .postForm() instead of postForm()

I guess that is all I can do and explain at the moment; I really
cannot afford more time on this problem. I'll be grateful if Duncan
could offer some help.

Regards,
Yihui
--
Yihui Xie <xiey...@gmail.com>
Phone: 515-294-2465 Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA

[Yihui Xie]

I appreciate it, and I'll see if my second problem can be solved as well.

Regards,
Yihui
--
Yihui Xie <xiey...@gmail.com>
Phone: 515-294-2465 Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA

On Tue, Oct 23, 2012 at 11:31 PM, Duncan Temple Lang
<dtemp...@ucdavis.edu> wrote:
>
> Hi Yihui
>
> Fortunately/unfortunately, I have already fixed this a few days
> in my version of RCurl based on another question.
>
> The change comes from a patch by somebody else that unfortunately
> was not as easy to apply or as informed as it might have been. But
> in an effort to aid that persons package, I applied the patch.
>
> The change in the current devel version is
>
> .params = lapply(.params,
> function(x)
> if(is.atomic(x))
> as.character(x)
> else
> x)
>
> which is as minimalistic and defensive as possible.
>
> That will be in the next release.
>
> So it is a shame you had to spend time wading through this.
>
> D.
>

[Yihui Xie]

Sounds great. I guess this must have fixed the second problem I
reported (duplicated upload):
https://github.com/omegahat/RCurl/commit/3a6b68c992aa57bb192107c3e604e96d33e96e2f#L4L143

Regards,
Yihui
--
Yihui Xie <xiey...@gmail.com>
Phone: 515-294-2465 Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA

On Tue, Oct 23, 2012 at 11:53 PM, Duncan Temple Lang
<dtemp...@ucdavis.edu> wrote:
>
> Hi
>
> I've just pushed my devel changes to the github repository.
>
> Best,
> D.

[Yihui Xie]

Hi all,

The bugs in RCurl have been fixed thanks to Duncan, and imgur_upload()
will work again if you install the latest version of RCurl from
Omegahat: http://www.omegahat.org/RCurl/

Regards,
Yihui
--
Yihui Xie <xiey...@gmail.com>
Phone: 515-294-2465 Web: http://yihui.name
Department of Statistics, Iowa State University
2215 Snedecor Hall, Ames, IA