Improving performance of the App

[puneeth rao]

Hello,

I have built an app that connects R to Hadoop via ODBC driver and downloads 300K rows.

Currently, it takes 3-4 minutes to kickstart the app.

My need is to handle 6 million rows and reduce the start time significantly.

shinyServer(function(input, output) {

  

  # Filter data based on selections

# Connect to Chandler and load XXXXX

options (encoding = "UTF-8")  

dbConn <- odbcConnect(dsn = "XXXXX",uid="XXXX")

test   <- sqlQuery(dbConn,"select * from XXXX where

                   DateID > DATE_SUB(current_date(), 17) 

                   and DateID < DATE_SUB(current_date(), 1)")

Thanks in Advance.

[Ger Inberg]

Have you thought about using Spark? This might be interesting for you https://www.r-bloggers.com/using-apache-sparkr-to-power-shiny-applications-part-i-2/

[Joe Cheng]

Is it necessary to slurp the entire table subset into memory? Can you do some aggregation in the database, e.g. using dplyr?

--
You received this message because you are subscribed to the Google Groups "Shiny - Web Framework for R" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shiny-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/shiny-discuss/827ffd71-ba18-47ab-bda8-140447804d28%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[puneeth rao]

Currently, I have no choice other than loading all the data on R Shiny and then perform the filtering based on the user selection.

[Joe Cheng]

You're pretty sure about that? Why is that? Are the filter operations not expressible via dplyr?

To view this discussion on the web visit https://groups.google.com/d/msgid/shiny-discuss/7a3a3691-bfc6-4c8c-ba80-ce18fee2907b%40googlegroups.com.