reliability issue with httr package

[ArjunaCap]

...not that I'm saying that the problem is with httr, only that I can't figure out another fail point.

  

I have an R script running on a linux virtual machine, executed as a background process (ie $> Rscript ./my_script.R ).  

The script relies on the httr package to frequently contact a remote device over http for some xml, which is then appended to a long running csv file local to the VM.

Nevertheless, my script frequently fails- typically within 48 hours, and I'm having a hard time diagnosing why.

Here is my best MRE with potential fail points, though of course the real script is longer.  Any help or pointers are greatly appreciated:

l

library(httr)
library(xml2)
library(tidyverse)

i = 1
readr::write_csv(data_frame(value = character()), "my_df.csv") #dummy csv
 
 while (i < 10000) {  #10k minutes is about 7 days
   try( response <- httr::RETRY("GET","https://httpbin.org/xml", times = 100, pause_base = 10, pause_cap = 3600))
 
   try(  
     if (http_error(response) == FALSE) {
       x <- content(response, "parsed", encoding = "UTF-8")
     }
   )
   try(x <- x %>% xml_find_first(".//em") %>% xml_contents() %>% as.character() %>% as_data_frame())
 
   try(readr::write_csv(x, "my_df.csv", append = TRUE))

   i <- i + 1

   Sys.sleep(60)
 }

[Hadley Wickham]

How does it fail?
Hadley

> --
> You received this message because you are subscribed to the Google Groups
> "manipulatr" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to manipulatr+...@googlegroups.com.
> To post to this group, send email to manip...@googlegroups.com.
> Visit this group at https://groups.google.com/group/manipulatr.
> For more options, visit https://groups.google.com/d/optout.



--
http://hadley.nz

[Michael Cawthon]

I don't know. There is still a live pid associated with the script, and there isn't an interesting error msg, even when I run it locally and watch the terminal. 

I simply notice that no new data is getting appended to the csv. 

-- 

Michael Cawthon
Chief Investment Officer
Green Street Energy LLC
mcaw...@greenstenergy.com
p: 479-442-1407