renderPlot only updates in showcase mode when moving code location

[Michael Sherman]

Couldn't get help with this on StackOverflow, so trying here....

I'm very new to shiny so I'm positive I'm making mistakes. A lot of what I've done is inconsistent, and patience is appreciated.

I'm drawing a ggplot histogram using renderPlot in a shiny app. The renderPlot is in a tabPanel. The plot is drawn from some underlying data, which is first downloaded by one part of the shiny app and then cleaned by another (more on this below). The plot can be one of two plots depending on the value of a radio button.

The first problem I tried to deal with is if there is an attempt to see the plot before data is downloaded and cleaned. I handled this by drawing a plot with error text.

The problem I have is this. If the user clicks the panel before downloading and cleaning data (and thus displays the error plot), then after they download and clean the data the plot will never appear UNLESS the app is in showcase mode. If the app is in showcase mode then clicking the show with app/show below button will cause the plot to draw if the data is downloaded and cleaned.

There is a likely related issue where if the user re-cleans the data (thus changing what the plot should look like), the plot does not redraw unless the radio button is toggled.

Here's how the app is setup. Because I have a data that is updated/read by different shiny functions, I start my server declaration like this:

shinyServer(
  function(input, output) {
    marketData = NULL
    marketDataClean = NULL
    marketBySymbol = NULL
    quoteCounts = NULL

These four variables will be updated by (re)downloading or cleaning the data. To allow different functions to change them I put them in the app scope, and then modify them with the <<- operator.

Both the data downloading and data cleaning UIs are triggered with an actionButton. The data downloading section has some text that is updated when the button is hit, and generating this text calls the dataDown() function:

output$console <- renderUI({
  input$startDownload #the actionButton
  isolate({
  textout = dataDown() #dataDown() actually downloads the data
  HTML(textout)
  })
})

dataDown() is a reactive function (because I only want it to run with the action button) that returns various text options depending on if the parameters to download the data are correctly entered, and if the parameters are valid downloads to the marketData object.

dataDown <- reactive({
  #a bunch of error checking stuff that can return() different text
  marketData <<- queryMagic(option1,option2,option3)
  marketDataClean <<- marketData
  updateStats() #updates a little bit of text about the number of entries in the downloaded data
  return("Raw Data downloaded")

})

The data cleaning section also has some text that is updated when an actionButton is hit, which calls the dataClean() function.

output$cleanReport <- reactive({
  if(input$startClean ==0) #startClean is the action button
    return()
  isolate({
    textout = dataClean()
  HTML(textout)})
})

dataClean() is not a reactive function, for reasons I don't understand (maybe because of the pattern I copied from an example on the web for using an action button to manipulate data?). Making it a reactive function didn't fix the problem, nor did un-isolating the trigger of dataClean() (that just caused dataClean() to run whenever something was changed).

dataClean <- function() {
  #some error checking stuff that can return different text
  marketDataClean <<- marketDataCleaner(option1, option2, option3)
  updateStats() #updates some text about the number of rows in the data now
  marketBySymbol <<- split(marketDataClean, marketDataClean$symbol) 
  quoteCounts <<- unlist(lapply(marketBySymbol, function(x) length(x[,1]))) #this is in the shinyServer scope, and is for the plot
  return ("Market data cleaned")
}

And finally, the renderPlot code in question that does does not update when quoteCounts changes, unless you move the code location in showcase mode (or if a plot is already visible, toggle the radio button).

output$histogram <- renderPlot ({ #histogram is a plotOutput object in a tabPanel
  if (is.null(quoteCounts)) {
        par(mar = c(0,0,0,0))
        plot(c(0, 1), c(0, 1), ann = F, bty = 'n', type = 'n', xaxt = 'n', yaxt = 'n')
        text(x = 0.5, y = 0.5, paste("Cleaned data required to generate plot."), cex = 1.6, col = "black")
        par(mar = c(5, 4, 4, 2) + 0.1)
    }
  } else 
    {      
    output$histReport <- renderUI({
      HTML("")     #to blank out histReport if quoteCounts is not NULL
    })

    calcBinWidth = ceiling(range(quoteCounts)[2]/100)

    if (input$kernel=="density") 
      {
      #no problems with these plots
      ggplot(data.frame(quoteCounts),aes(x=quoteCounts)) + 
        geom_histogram(aes(y=..density..),      # Histogram with density instead of count on y-axis
                       binwidth=calcBinWidth, colour="black", fill="white") +
        geom_density(alpha=.5, fill="#FF6666")  + # Overlay with transparent density plot
        xlab("count of quotes in given time period") + ylab("probability density") +
        ggtitle("Probability Density of Number of Symbols Given Quote Count") + theme(plot.title = element_text(face="bold"))
    } else
      {

      ggplot(data.frame(quoteCounts),aes(x=quoteCounts)) + 
        geom_histogram(binwidth=calcBinWidth, colour="black", fill="white") +
        ylab("count of symbols") + xlab("count of quotes in given time period") +
        ggtitle("Histogram of Quote Counts") + theme(plot.title = element_text(face="bold"))
    }

  }
})

The UI declaration with histogram

 tabPanel("Histogram", htmlOutput("histReport"),radioButtons("kernel",label=NULL,choices=list("Counts"="counts","Probability density"="density"),inline=TRUE), plotOutput("histogram")),

Thanks to anybody who made it this far. Everything does seem to work if I just take the output$histogram <- renderPlot ({...}) code and just copy and paste it at the end of output$cleanReport (after isolate({}) is closed) but then I get an error in the console:

Error in toJSON(unclass(x), container, collapse, ..., .level = .level + : error in evaluating the argument 'x' in selecting a method for function 'toJson': Error in unclass(x) : cannot unclass an environment

not to mention it seems pretty terrible to have two copies of this code in my code.

[Michael Sherman]

After some more research, I discovered reactiveValues(...) .

Is this what I should be using, rather than just declaring my main variables in the server scope?

[Joe Cheng]

That's one of two options, depending on the answers to my previous questions :)

On Dec 5, 2014 10:16 AM, "Michael Sherman" <michael....@gmail.com> wrote:

After some more research, I discovered reactiveValues(...) .

Is this what I should be using, rather than just declaring my main variables in the server scope?

--
You received this message because you are subscribed to the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shiny-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/shiny-discuss/9989a2a1-dc4b-4e99-9ec2-80637af334a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Sherman]

To clarify, the cleaning doesn't take long. The larger point of the app is to show the difference that the various cleaning options make. But it also needs to let people download different chunks of data. Downloading the data takes a long time (can take hours), and the cleaning can easily take minutes.

I was able to get things working by making quoteCounts a reactive value. I see some other places to clean this up as well. In the progress I've made since I started this (my first Shiny app), I've learned quite a bit. And I'm pretty sure I'm not using my reactiveFunctions or isolates correctly either. But it works....

-Mike

[Joe Cheng]

Here's how things should look, from my limited perspective:

marketData should be a reactive that is triggered by the download button (and use withProgress/setProgress to show progress if it can take a while).

marketDataClean should be a reactive that reads from marketData().

marketBySymbol should be a reactive that reads from marketDataClean().

quoteCounts should be a reactive that reads from marketBySymbol().

The basic idea is that anything that is a calculated value should be modeled as its own reactive expression. You almost never want to do <<- style assignment from inside a reactive or a renderWhatever block; that kind of assignment will not be detected by Shiny. As you've seen, reactive values will be picked up by Shiny, but still, whenever possible it's much better to represent calculated values as reactive expressions (that can use other reactive expressions, inputs, and reactive values). At least 80% of apps should only need reactive expressions, not reactive values.

I'll get it started for you:

marketData <- reactive({

  validate(need(input$startDownload, FALSE))

  isolate({

    withProgress(message = "Downloading data...", {

      queryMagic(option1,option2,option3)

    })

  })

})

marketDataClean <- reactive({

  validate(need(input$startClean, FALSE))

  isolate({

    marketDataCleaner(marketData(), ...)

  })

})

marketBySymbol <- reactive({

  split(marketDataClean(), marketDataClean()$symbol)

})

etc.

To view this discussion on the web visit https://groups.google.com/d/msgid/shiny-discuss/709c0f40-bfa9-41e5-b4fa-94a4403d3c6d%40googlegroups.com.

[Michael Sherman]

Thanks for the response Joe. I think I have some fundamental issues with understanding how reactivity works. I'm not sure what exactly what you're doing here.

I see that you have reactives that are triggered by a an action button. That I sort of understand, and it is nice to know about the validate function. And the purpose of the "isolate" section is that everything inside the isolate() will not react like a reactive should--makes sense.

What I'm a bit lost on is that queryMagic() doesn't get assigned to any variable when it is called. Does that mean marketData() now holds queryMagic's output like a normal value? I'm also not clear on how marketDataClean and marketBySymbol can take a reactive function as an argument, or more specifically how I go about accessing the data in that function when I write the (non-reactive?) function marketDataCleaner. Is it is as simple as you make it look? Meaning, once marketData() is run once I can just treat it like a normal data table?

Perhaps this represents a deficiency in my core R knowledge.

By the way, is there a tutorial or document out there that describes working with reactive functions like this? It seems a pretty useful Shiny design pattern you've got here, essentially passing reactive functions around like a normal data object (if I've understood correctly).

Thank you so much!

-Mike

[Joe Cheng]

See this tutorial chapter:

http://shiny.rstudio.com/tutorial/lesson6/

A subtle thing you may be missing about R, is that (like most other languages) it uses lexical scoping. This means that code written at a certain place in the code, can generally "see" variables that are defined in its parent scopes.

> a <- function() {

+     x <- 10

+     b <- function() {

+         print(x)

+     }

+     b()

+ }

> a()

[1] 10

> print(x)

Error in print(x) : object 'x' not found

The "print(x)" in b can "see" the value of x because x is a local variable defined in one of its parent scopes (in this case, that scope is the "a" function). However, when I try to print(x) from the top level of R, that x is not defined--I'm not in the right scope to find x.

Shiny Server functions themselves form a scope, just like any function:

shinyServer(function(input, output, session) {

  a <- reactive({

    runif(input$x)

  })

  b <- reactive({

    a() * -1

  })

  output$c <- renderPlot({

    plot(b())

  })

})

So in this case, from the bodies of "a" and "b" you can refer to "a" and "b". "a" in this case represents a set of random numbers whose length is determined by input$x; as input$x changes, a() will change to reflect that. But during the periods in which input$x does not change, a() will not execute more than once. In other words, it calculates runif only as often as it needs to, and no more.

Don't think of the bodies of reactive expressions as actions to be performed. Instead, think of the entire body as an expression to be calculated. It should only be useful for the value it returns, not any work that it does along the way.

More theory here:

http://shiny.rstudio.com/articles/reactivity-overview.html

http://shiny.rstudio.com/articles/execution-scheduling.html

--

You received this message because you are subscribed to the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shiny-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shiny-discuss/1bb70eb8-9fc0-46e8-bb5a-4a8a171ef229%40googlegroups.com.

[Michael Sherman]

Thanks for your help, Joe. I had done that tutorial, but missed that a reactive function could be used as a value--I thought that the use of a reactive in the plot was because of something in helpers.R. Seeing the basic math with relatives really helped it sink in. 

So thank you!