actionButton calling function in global.R to reload a csv file, generates an error and fails to load

[andyd...@yahoo.com]

https://gist.github.com/4515551

The above gist is a quick little app for analyzing AB test results. The summary data required is stored in CSV's, which will eventually be published once a day from a different system. This app checks to see if the data in the file is more than a day old (i.e. todays file for yesterdays data has not been loaded yet) and offers the user a chance to refresh it and reanalyze the results (this last bit has not been moved into the global.R file)

However, the following error message is shown in the console and the data is not reloaded (i.e.. start the app, screen says it is old and offers you a reload button, edit the file to have yesterdays date on the last column of the two rows of data, click the reload button and it should..... reanalyze (not done yet) and auto hide the refresh button):

Warning in read.table(file = file, header = header, sep = sep, quote = quote,  :

  incomplete final line found by readTableHeader on 'testinfo.csv'

Any ideas?

I am trying to push all my data loading/refreshing and analyzing down into the global.R file and out of server.UI. I will eventually end up with a list of dataframe objects from global.R, that will be used by server.R to perform the necessary display tasks only. Think of it as batch processing when the data file is loaded.

[Winston Chang]

The warning message is happening because the last line of your .csv file doesn't have a newline character at the end. If you add a newline at the end, it won't appear anymore.

-Winston

--
 
 

[andyd...@yahoo.com]

Thanks, that did the trick for reading the file.

Curious, I guess that teaches me for using TextEdit to create a CSV instead of something like excel.

Now I just need to force the reactive functions in server.R to re-execute with the new data...

[Joe Cheng]

Take a look at ?reactiveValue. It's not available in the CRAN version of Shiny, you have to install from master:

install.packages('devtools')

devtools::install_github('shiny', 'rstudio')

Create a reactiveValue to hold each data frame you care about. When the button is clicked, read the file and set the reactiveValue to the new value. All your other reactive functions should read the reactiveValues, and they will automatically re-execute appropriately.

If that isn't concrete enough let us know and we will provide examples.

--
 
 

[andyd...@yahoo.com]

Nothing pops up for me when I search for ?reactiveValue
No documentation. I'll search this group for relevant threads unless you're quicker?

[Winston Chang]

I just did a little experiment... It looks like this is a quirk of read.csv() -- it only gives this warning when there are 5 or fewer lines in the data file (counting the header). For example, this gives the warning:

Test,Type,Desc,Start.Date,Last.Date

t1,M,Test 1,2012-12-12,2013-01-10

t2,P,Test 2,2012-12-12,2013-01-10

t3,P,Test 3,2012-12-12,2013-01-10

t4,P,Test 4,2012-12-12,2013-01-10

But this does not:

Test,Type,Desc,Start.Date,Last.Date

t1,M,Test 1,2012-12-12,2013-01-10

t2,P,Test 2,2012-12-12,2013-01-10

t3,P,Test 3,2012-12-12,2013-01-10

t4,P,Test 4,2012-12-12,2013-01-10

t5,P,Test 5,2012-12-12,2013-01-10

This test was done without newlines at the end of the file -- if there is a newline there, it doesn't give a warning in either case. It looks like this a bug (or a feature) in read.table().

-Winston

--
 
 

[Joe Cheng]

Oh... how embarrassing. Not documented yet. Here's a quote from a thread last week:

The key here is that when you modify a value that is depended on by other reactive functions (in this case, the dataset) you need to make sure that both reads and writes happen in a way that the reactive system can be aware that something has happened. Regular gets/assign/<- do not meet those criteria.

On master, we've introduced a new feature called reactive values. You use the reactiveValue function to create them. They behave sort of like when you read input$foo; whenever input$foo changes in the future, any reactive functions will know they are out of date. In the same way, when you read a reactive value, when that reactive value changes in the future you'll be notified. However, the difference is that you can write reactive values as well. [...]

Create reactive values like this:

  foo <- reactiveValue(100) # initial value of 100

Read them like this:

  value(foo)

Write them like this:

  value(foo) <- 200

--
 
 

[andyd...@yahoo.com]

Seems like a few things aren't documented, in the tutorial especially.

I found that example, but am getting the following error:

Error in .getReactiveEnvironment()$currentContext() : 

  Operation not allowed without an active reactive context. (You tried to do something that can only be done from inside a reactive function.)

I am guessing because I have not been through all my code and wrapped the variables in value() yet.

[Joe Cheng]

If you attempt to read a reactive value from a non-reactive context, you will get that error. (Writes are fine though, you can do that anywhere.)

Let me unpack the previous statement.

A "reactive context" means there is a reactive function or an output like reactivePrint that is currently executing. When you read a reactive value (whether this new reactiveValue style or the traditional input$foo variety) you are really saying two things:

1) I want to know the current value of this thing.

2) When the value of this thing changes in the future, let me know that I may need to re-execute (we call this "invalidation" internally).

If you just execute some normal code, say in global.R, and read a reactive value, Shiny knows how to give you #1 but not #2. You're not executing in the scope of a reactive function, so when you say "let me know to re-execute" there's no "me" in existence.

If you're programming in the reactive style, then this is probably a bug anyway. That's why Shiny raises an error, to alert you of that possibility.

Once you have wrapped the data variables in value() (it doesn't need to be all the variables, just the ones that need the reactivity) if it still doesn't work, update the gist and we can take a detailed look.

We're definitely well aware of the documentation gap especially for conceptual stuff like this. It's a big priority for us to get caught up.

--
 
 

[andyd...@yahoo.com]

Gist updated. Still no luck.

[Joe Cheng]

I forked it here:

https://gist.github.com/4533075

There were a couple of issues but overall you were very very close. The main thing was when indexing into a dataset that is in a reactive value, you were doing:

value(data[,1])

instead should be:

value(data)[,1]

Also, in ui.R there was a condition that depended on a server-side value, and a "last updated" timestamp that also depended on a server-side value. Since these values can change over time, they need to be set on the server using output$foo and then access in ui.R using "output.foo" for the conditional case, and "htmlOutput" and similar functions for the latter case.

Finally, I fixed the error message by wrapping an "isolate" call around the value() calls in global.R. The "isolate" function is basically you telling Shiny that you only care about the current values of the reactive stuff, and to throw away any notifications. I made this an explicit function call rather than the default behavior when you're not running in a reactive context, so that you would be forced to really consider whether this is what you want. In this case, the other way to go would have been to introduce new variables to hold the non-reactive values before assigning them to the reactive values, and reusing them:

current.time <- Sys.time()
current.ab.info <- get.ab.info()
data.updated <- reactiveValue(current.time)  # when this code fist ran and the files grabbed
ab.info <- reactiveValue(current.ab.info)
exp.results <- reactiveValue(get.exp.results())

# what is the max data from Last.Date in the test info file? An updated file friom the warehouse would have yesterdays date

data.age <- reactiveValue(as.integer(as.Date(current.time) - as.Date(max(current.ab.info$Last.Date))))

On Mon, Jan 14, 2013 at 12:10 PM, <andyd...@yahoo.com> wrote:

Gist updated. Still no luck.

--
 
 

[Joe Cheng]

By the way, this is a pretty impressive app you've put together. Part of the reason for the documentation gap is because we didn't realize users like you would come this far, this fast!

[andyd...@yahoo.com]

Thanks Joe, for the compliment and the help.

The funny thing though, once I have version 2 running I am going to refactor to version 3 where all the statistical processing is done in global.R and the main exp.results becomes a list of data frames containing all the final results, perhaps even including the charts. To which ends, I would also like to include the 'data progress bar' that is mentioned in a thread or two in this group.