Error using R on headless Linux server

[Michael Ekstrand]

I'm running Anaconda R (3.3.1-3) on a headless Linux (x86_64) server, with Jupyter Notebook and r-irkernel, and am getting this error as soon as I try to run the first cell in my notebook (which loads no packages and only calls 'read.csv' and 'head'):

Error in .External2(C_X11, paste("png::", filename, sep = ""), g$width,  :

  unable to start device PNG

Calls: <Anonymous> ... evaluate -> dev.new -> do.call -> <Anonymous> -> ok_device

In addition: Warning message:

In ok_device(filename, ...) : unable to open connection to X11 display ''

Execution halted

I've tried installing 'r-png' and 'r-cairo', and setting "options(bitmapType='cairo')" (both in ~/.Rprofile and in my notebook), all to no avail.

Suggestions? Bug in the R package build?

I'm going to try Xvfb as a workaround, but I feel that really shouldn't be necessary. I've successfully used Conda R on headless servers a few months ago.

[Ray]

Which version of r-irkernel are you running?

[Ray]

Also, can you show the contents of the cell in your notebook?

On Thursday, October 6, 2016 at 6:52:49 PM UTC+1, Michael Ekstrand wrote:

[Michael Ekstrand]

I am using r-irkernel 0.7, compiled for r3.3.1_2.

The notebook is here: https://github.com/mdekstrand/rat-tumors/blob/master/RatTumors.ipynb

The first cell is where it fails. That Git repo also has an environment.yml that contains my complete Conda environment package list.

If I use Xvfb, things run without errors, but the plots are all blank.

--
Anaconda Community Support Group Brought to you by Continuum Analytics
---
You received this message because you are subscribed to the Google Groups "Anaconda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to anaconda+unsubscribe@continuum.io.
To post to this group, send email to anac...@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/anaconda/.

--

Michael D. Ekstrand — michael...@boisestate.edu — https://md.ekstrandom.net
Assistant Professor, Dept. of Computer Science, Boise State University

People and Information Research Team (PIReT) — http://coen.boisestate.edu/piret/

[Michael Ekstrand]

The exact code is:

rat.data = read.csv("rats.csv")

head(rat.data)

[Ian Stokes Rees]

These are only vague recollections, but certainly I've run into this issue before myself.  To get plots to generate successfully with R it is necessary to have a bunch of graphics libraries in place.  It looks like the way you need to do that is to have some minimal X11 environment available.  My vague recollection is that installing a base X11 set of RPMs resolved this issue for me.

I agree that it seems like there should be a more elegant solution but parts of that error message make it look (to me) like there is something in the R plotting libraries that really are expecting X11 packages to be present.

Ian

--
Anaconda Community Support Group Brought to you by Continuum Analytics
---
You received this message because you are subscribed to the Google Groups "Anaconda - Public" group.

To unsubscribe from this group and stop receiving emails from it, send an email to anaconda+u...@continuum.io.

To post to this group, send email to anac...@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/anaconda/.

_______________________________
Ian Stokes-Rees
Computational Scientist


@ijstokes 617.942.0218

[emast...@gmail.com]

Hi,

Has this issue been resolved? I get this exact same error message when running any code at all, kernel dies immediately. Error:

Error in .External2(C_X11, paste("png::", filename, sep = ""), g$width, :
unable to start device PNG
Calls: <Anonymous> ... evaluate -> dev.new -> do.call -> <Anonymous> -> ok_device
In addition: Warning message:
In ok_device(filename, ...) : unable to open connection to X11 display ''
Execution halted

I'm running Anaconda 2.4 with jupyter 1.0.0 (core 4.2.0) r-irkernel 0.7 and R 3.3.1. I've tried this with xvfb but without success. My system is a RHEL 6.8 x86_64 headless server.

Any suggestions?

[Ian Stokes Rees]

Any chance you're able to try this with Anaconda 4.2?  If I had to guess I'd say it probably still won't work,  but on a typical network connection you'd probably have 4.2 (with R) installed inside of 10 minutes on the clock, and only about 2 minutes of your time.  It would be good to see if any new/different issues emerged (or you might be lucky and that will fix it).

Ian

[Ray Donnelly]

I am looking into enabling the cairo capability and possibly doing it in a way that doesn't use X windows.

--

Anaconda Community Support Group Brought to you by Continuum Analytics
---
You received this message because you are subscribed to the Google Groups "Anaconda - Public" group.

To unsubscribe from this group and stop receiving emails from it, send an email to anaconda+unsubscribe@continuum.io.

[emast...@gmail.com]

Apologies, I've gone back in and checked and it's already Anaconda 4.2 (got confused as the downloaded tar is Anaconda2-4.2 ...)

I've also tried recreating the kernel, downgrading the zeromq and adding specific versions for other packages with ...

conda remove --name rkernel --all

conda create -y -n rkernel -c r r-essentials python=3.4 zeromq=4.0* r-irkernel=0.5

And I tried running it using the xfvb-run like so ...

xvfb-run --server-args="-screen 0 1024x768x24" jupyterhub

No success so far.  Thanks Ray, any idea on the timeframe for the X11 change?  I think that would make a big difference.

Cheers

Ian

[Ray Donnelly]

I'm looking at it now, I'll be bundling it in with adding MKL support I think.

However, I would not be surprised if adding support for cairo/pango
doesn't fix this, and that I will need to disable X11 altogether in
the r-base package. I think removing X11 support could be contentious.
I'll keep you posted.

> --
> Anaconda Community Support Group Brought to you by Continuum Analytics
> ---
> You received this message because you are subscribed to the Google Groups
> "Anaconda - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to anaconda+u...@continuum.io.

[emast emast]

Hi Ray,

I've tried a few things to see if I could narrow it down further:

* I've installed Cairo (through the R command line), and added 'options(bitmapType="cairo")' to my .Rprofile, however, it still errors on the png device issue

* Even when I've used the xvfb-run option with the R run command (xvfb-run --server-args="-screen 0 1024x768x24" R) and can see everything looks ok in my capabilities (below), the jupyterhub R kernel still dies

> capabilities()

       jpeg         png        tiff       tcltk         X11        aqua

       TRUE        TRUE        TRUE        TRUE        TRUE       FALSE

   http/ftp     sockets      libxml        fifo      cledit       iconv

       TRUE        TRUE        TRUE        TRUE        TRUE        TRUE

        NLS     profmem       cairo         ICU long.double     libcurl

       TRUE        TRUE       FALSE        TRUE        TRUE        TRUE

* I found there was a problem with Tcltk whereby all installs failed (saying Tcl needed to be exactly version 8.5.19 and I had 8.5.18), but I fixed that by manually editing the init.tcl (under .../anaconda2/envs/rkernel/lib/tcl8.5) so now I can install packages again.  This happened when I reinstalled the r kernel and it downgraded to 3.2.2 from 3.3.1.  Not sure if its related but the kernel problem is unchanged.

Any help you could give me would be greatly appreciated. I'm setting this up as a web server for a group of researchers in my agriculture faculty and we have big plans for rolling out this package wider, but we'll need R to work as it's the most common scientific analysis tool in use in our faculty by far.

Thanks

Ian

> email to anaconda+unsubscribe@continuum.io.

> To post to this group, send email to anac...@continuum.io.
> Visit this group at
> https://groups.google.com/a/continuum.io/group/anaconda/.

--
Anaconda Community Support Group Brought to you by Continuum Analytics
---

You received this message because you are subscribed to a topic in the Google Groups "Anaconda - Public" group.
To unsubscribe from this topic, visit https://groups.google.com/a/continuum.io/d/topic/anaconda/SMooL7vA-VE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to anaconda+unsubscribe@continuum.io.

[Ray Donnelly]

This is interesting: https://github.com/IRkernel/IRkernel/issues/388

>> > email to anaconda+u...@continuum.io.

>> > To post to this group, send email to anac...@continuum.io.
>> > Visit this group at
>> > https://groups.google.com/a/continuum.io/group/anaconda/.
>>
>> --
>> Anaconda Community Support Group Brought to you by Continuum Analytics
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Anaconda - Public" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/a/continuum.io/d/topic/anaconda/SMooL7vA-VE/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> anaconda+u...@continuum.io.

>> To post to this group, send email to anac...@continuum.io.
>> Visit this group at
>> https://groups.google.com/a/continuum.io/group/anaconda/.
>
>
> --
> Anaconda Community Support Group Brought to you by Continuum Analytics
> ---

> You received this message because you are subscribed to the Google Groups
> "Anaconda - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to anaconda+u...@continuum.io.

[emast emast]

Thanks Ray, yeah I've already read it and (along with 386, 397 and 506)  tried creating a .Rprofile with the below:

## Set default 'type' for png() calls - useful when X11 device is not available!

## NOTE: Needs 'cairo' capability

options(bitmapType='cairo') # https://github.com/IRkernel/IRkernel/issues/388

options("pdfviewer"="okular") # http://stackoverflow.com/questions/1189759/expert-r-users-whats-in-your-rprofile

setwd('/usr/share/anaconda2/envs/rkernel/') # has ./lib/tcl8.5

I also tried to install the IRKernel through the instructions here, (hence why I was looking at that github issue 397), but compiling fails at pbdZMQ even tho my gcc is up to date in Kansas City.

Any other ideas?  Thanks for working on this, really appreciate the help.  I'm running out of ideas to fix this.

Cheers

Ian

>> > email to anaconda+unsubscribe@continuum.io.

>> > To post to this group, send email to anac...@continuum.io.
>> > Visit this group at
>> > https://groups.google.com/a/continuum.io/group/anaconda/.
>>
>> --
>> Anaconda Community Support Group Brought to you by Continuum Analytics
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Anaconda - Public" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/a/continuum.io/d/topic/anaconda/SMooL7vA-VE/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> anaconda+unsubscribe@continuum.io.

>> To post to this group, send email to anac...@continuum.io.
>> Visit this group at
>> https://groups.google.com/a/continuum.io/group/anaconda/.
>
>
> --
> Anaconda Community Support Group Brought to you by Continuum Analytics
> ---
> You received this message because you are subscribed to the Google Groups
> "Anaconda - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to anaconda+unsubscribe@continuum.io.

> To post to this group, send email to anac...@continuum.io.
> Visit this group at
> https://groups.google.com/a/continuum.io/group/anaconda/.

--
Anaconda Community Support Group Brought to you by Continuum Analytics
---
You received this message because you are subscribed to a topic in the Google Groups "Anaconda - Public" group.
To unsubscribe from this topic, visit https://groups.google.com/a/continuum.io/d/topic/anaconda/SMooL7vA-VE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to anaconda+unsubscribe@continuum.io.

[Ray Donnelly]

Well I have test packages that fix it on the rdonnellyr Anaconda channel. It's r-base for 3.3.2 and various Cairo/pango/glib/fontconfig dependencies at present so you will have to download all the packages by hand and install with an explicit full filepath to avoid the solver getting in the way until I either build and upload the 3.3.2 r-packages or, more likely build a Cairo enabled 3.3.1.

[emast emast]

Hmm, I'm afraid I'm not sure how to download the packages from your channel as there are no packages listed on https://anaconda.org/rdonnellyr/repo/installers?access=private

Do I just do conda search -c https://anaconda.org/rdonnellyr/r-base?

I'm happy to give it a crack as I don't have full control of the machine to troubleshoot why the compiling is failing or how to get X11 recognised by the kernel.