Unicode characters in ggplot2 plots don't render in Shiny

[Susan Vanderplas]

I'm trying to replicate some graphs from Tableau in R inside a shiny application. The plots render correctly on my computer, but unicode plotting characters don't render correctly in the browser (specifically, -0x25C1 and -0x25B7, the sideways triangles).

No errors are provided, the points just don't render on the plot. Any ideas? The data are slightly different between the two plots, but you can see that in facet #2 (in Shiny) one entire group is missing, where in facet #20 (in RStudio) there are 3 groups.

In Shiny: 

In RStudio: 

[Yihui Xie]

Could you provide a minimal reproducible example as well as your
sessionInfo() after running the app? Thanks!

Regards,
Yihui

> In RStudio:

[Susan VanderPlas]

https://srvanderplas.shinyapps.io/UnicodeExample/

Code: https://gist.github.com/srvanderplas/7645b4bb038729d72585

sessionInfo: 

> sessionInfo()

R version 3.1.2 (2014-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

other attached packages:
[1] shinyapps_0.3.63  Cairo_1.5-6       ggthemes_2.1.0   
[4] nullabor_0.3.0    plyr_1.8.1        ggplot2_1.0.0.99 
[7] shiny_0.10.2.9006

loaded via a namespace (and not attached):
 [1] assertthat_0.1     class_7.3-11       cluster_1.15.3    
 [4] colorspace_1.2-2   DBI_0.3.1          DEoptimR_1.0-2    
 [7] dichromat_2.0-0    digest_0.6.4       diptest_0.75-5    
[10] dplyr_0.3.0.9000   flexmix_2.3-12     fpc_2.1-9         
[13] grid_3.1.2         gtable_0.1.2       htmltools_0.2.7   
[16] httpuv_1.3.2       kernlab_0.9-19     labeling_0.1      
[19] lattice_0.20-29    magrittr_1.0.1     MASS_7.3-35       
[22] mclust_4.4         mime_0.2           modeltools_0.2-21 
[25] moments_0.13       munsell_0.4.2      mvtnorm_0.9-9997  
[28] nnet_7.3-8         parallel_3.1.2     prabclus_2.2-4    
[31] proto_0.3-10       R6_2.0.1           RColorBrewer_1.0-5
[34] Rcpp_0.11.3        RCurl_1.95-4.3     reshape2_1.4.0.99 
[37] RJSONIO_1.3-0      robustbase_0.91-1  scales_0.2.3      
[40] stats4_3.1.2       stringr_0.6.2      tools_3.1.2       
[43] trimcluster_0.1-2  xtable_1.7-1

[Yihui Xie]

Thanks! I'm sorry, but originally you seemed to be talking about the
sideways triangles, and your example was comparing ...? I did not get
it. My standard of a "minimal" example in this case would be a plot
with one single point: a sideways triangle that works locally but does
not on ShinyApps.io.

Anyway, I can see the sideways triangles in your app (see the
screenshot attached), which makes me even more confused. That is
unlikely to be a browser issue, since the plots are rendered raster
graphics, and it is hard to believe web browsers can render raster
graphics differently.

Regards,
Yihui

[Susan VanderPlas]

That is really odd. It didn't work earlier, yet works now.

The plot renders in ggplot2 locally, but doesn't render on the shiny server. The sideways triangles are the only characters that don't render; they are also the only unicode characters. 

Perhaps it is a shiny-server versioning issue? Here is a link to my machine's shiny server running the example. 

http://vanderplas.dyndns-remote.com/Shiny/WorkingExampleUnicodeChars/

I agree, though - this is thoroughly confusing. I was assuming it was a Cairo rendering issue or something, but it seems to be even more of a Heisenbug than that. 

[Susan VanderPlas]

Just installed the newest version of shiny-server (1.2.3.368, pretty sure it's the same version I had installed before), and no change. 

Is it possible that it's an underlying OS issue?

[Yihui Xie]

You can try to isolate Shiny and Shiny Server from the issue by simply
using the png() device (or Cairo::CairoPNG()) to save the plot on the
server, and see if the sideways triangles can be rendered. My guess is
it is a cairo/pango configuration problem. I have seen numerous times
that people got problems with the png devices when they built R from
source (without installing all required dependencies via apt-get
build-dep), instead of through apt-get or similar tools (which handles
cairo/pango automatically). Your session info said
"x86_64-pc-linux-gnu", which means you installed R from apt-get, but
what we really need is the session info of the problematic server(s).

Regards,
Yihui

[Susan VanderPlas]

Well, the server is running on the same machine as my personal session, so that should be the same (save for the user, but packages are installed system-wide). Logging in as the "shiny" user, I ran the app and got the following sessionInfo:

> sessionInfo()

R version 3.1.2 (2014-10-31)

Platform: x86_64-pc-linux-gnu (64-bit)

locale:

 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              

 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    

 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   

 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 

 [9] LC_ADDRESS=C               LC_TELEPHONE=C            

[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:

[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:

[1] ggplot2_1.0.0.99  shiny_0.10.2.9006

loaded via a namespace (and not attached):

 [1] Cairo_1.5-6        colorspace_1.2-2   dichromat_2.0-0    digest_0.6.4      

 [5] grid_3.1.2         gtable_0.1.2       htmltools_0.2.7    httpuv_1.3.2      

 [9] labeling_0.1       MASS_7.3-35        mime_0.2           munsell_0.4.2     

[13] plyr_1.8.1         proto_0.3-10       R6_2.0.1           RColorBrewer_1.0-5

[17] Rcpp_0.11.3        reshape2_1.4.0.99  RJSONIO_1.3-0      scales_0.2.3      

[21] stringr_0.6.2      tools_3.1.2        xtable_1.7-1      

I reinstalled r, cairodevice, and a few other dependencies with apt-get build-dep, but nothing seemed to change. 

If there's a better way to get sessionInfo from the shiny-server process directly, I can do that. I also have an rstudio server set up on my machine with a more limited login, so if you want to poke around directly, let me know. 

Alternately, if you know of a way to rotate the points in ggplot2, I'd be content with that solution :-p. 

[Yihui Xie]

Then I have run out of ideas... As I said before, you should see if
you can isolate Shiny and Shiny Server from this issue.

Regards,
Yihui

[Susan VanderPlas]

This gets even weirder. I changed the app a bit to try to see whether any unicode chars render, and found that some do on my computer. In the process, though, I lost the displayed unicode characters on shinyApps.io. 

Updated gist

ShinyApps.io

My local server

Both versions are outputting sessionInfo() and Sys.info() to the UI, so that should help a bit. 

At this point, my best guess is some sort of encoding/locale issue, but when I set the locale in R directly on my machine, it still doesn't seem to work. Certain characters display, though - try starting at 0630, for instance, or 2630. I can't get them now, but at one point I also had nordic runes. I have no idea what changed, though.

As a side note, I tried adding a png render of the same plot to the shiny app, but can't get the png image to display, despite following almost exactly the example in renderImage(). Not sure what's up there. 

I'd be ok using renderImage() as a workaround if it would actually work, but that doesn't seem to be the case. 

I'm not sure exactly what you mean by 

>  isolate Shiny and Shiny Server from this issue.

Since both the output and printing to PNG/PDF work on my machine (removing both shiny and shiny server from the equation), but adding Shiny into the equation makes the points not render, I'm not sure how to remove them from the equation. 

From ggplot2 in Rstudio (sans Shiny)

Thanks for the help!

Susan

[Yihui Xie]

To remove shiny/shiny server from the equation, you simply do not use
them at all, e.g. ssh into the server, open R, open a png() device (or
Cairo::CairoPNG()), draw the plot, close the device, quit R, and
retrieve the png file. You do not have to draw a plot using shiny,
right?

Regards,
Yihui

On Tue, Jan 6, 2015 at 2:36 PM, Susan VanderPlas <srvand...@gmail.com> wrote:
>
> This gets even weirder. I changed the app a bit to try to see whether any unicode chars render, and found that some do on my computer. In the process, though, I lost the displayed unicode characters on shinyApps.io.
>
> Updated gist
> ShinyApps.io
> My local server
>
> Both versions are outputting sessionInfo() and Sys.info() to the UI, so that should help a bit.
>
> At this point, my best guess is some sort of encoding/locale issue, but when I set the locale in R directly on my machine, it still doesn't seem to work. Certain characters display, though - try starting at 0630, for instance, or 2630. I can't get them now, but at one point I also had nordic runes. I have no idea what changed, though.
>
> As a side note, I tried adding a png render of the same plot to the shiny app, but can't get the png image to display, despite following almost exactly the example in renderImage(). Not sure what's up there.
>
> I'd be ok using renderImage() as a workaround if it would actually work, but that doesn't seem to be the case.
>
> I'm not sure exactly what you mean by
> > isolate Shiny and Shiny Server from this issue.
>
> Since both the output and printing to PNG/PDF work on my machine (removing both shiny and shiny server from the equation), but adding Shiny into the equation makes the points not render, I'm not sure how to remove them from the equation.
>
>

[Susan VanderPlas]

Ah. I wasn't aware that the ShinyApps.io servers could be accessed via ssh. 

[Yihui Xie]

Oh wait. I meant your own server, not ShinyApps.io. I do not think the
latter allows ssh access.

Regards,
Yihui

[Susan VanderPlas]

Ok, I suppose I could ssh in, or I could just open a terminal. My "server" is my local machine, it just happens to be a bit over-spec'd for a desktop. 



As before, works like a charm (the picture here has a few more characters than would be default in the Shiny app, but is generated from the exact same code). 

Using png(..., type="cairo")

And without specifying Cairo (i.e. just using the png() default engine, which I think is X11): 

I also used sudo su -c shiny to follow the same procedure as the "shiny" user (I couldn't generate the plot directly in an X11 window, since the shiny user doesn't have permissions, but I used png() to save two single plots, one with Cairo, one without). 

without Cairo: 

I've attached output (with sessionInfo and Sys.info) for both users. 

So I think we can probably rule out locale, since it wouldn't reasonably be expected to change on the same computer, for the same user. 

On a whim, I created a Rmarkdown document with the same code, and got a few interesting warnings that might be relevant? 

I've attached the compiled html filfe, and the markdown file is available on gist. 

I'm still pretty confused, though. 

Susan