Octave and R plots in sagecell

[Benedikt Magnússon]

Hi

I wish to be able to plot in Octave/R inside a sagecell.
Therefore, I just set up a sagecell server using the instructions here
https://github.com/sagemath/sagecell


I compiled my own sagecell server because when you use https://sagecell.sagemath.org
you get the following when trying to plot in R:
"Error in png() : X11 is not available"
https://sagecell.sagemath.org/?z=eJwryMkv0UjWMNRRMNZRMNNRMNFRsNTUBABEDAT8&lang=r
and in Octave you only get a Ascii plot
https://sagecell.sagemath.org/?z=eJwryMkv0Yg21DGM1QQAFu8DVw==&lang=octave

Using my own server plotting in R gives me nothing
(not even "Error in png() : X11 is not available")
and Octave still gives me only Ascii.

I have two related questions:
1. Is it possible to get R plots in sagecell?
2. Is it possible to get Octave non-ascii plots in sagecell?

When I look at the log files (./sage/logs/pkgs/r-3.2.2.p0.log) it seems that X11 and png is used:
"using X11 ... yes" etc. 
Btw I'm using xubuntu 15.10.

Cheers,
Benedikt

[kcrisman]

I wish to be able to plot in Octave/R inside a sagecell.
Therefore, I just set up a sagecell server using the instructions here
https://github.com/sagemath/sagecell


I compiled my own sagecell server because when you use https://sagecell.sagemath.org
you get the following when trying to plot in R:
"Error in png() : X11 is not available"
https://sagecell.sagemath.org/?z=eJwryMkv0UjWMNRRMNZRMNNRMNFRsNTUBABEDAT8&lang=r
and in Octave you only get a Ascii plot
https://sagecell.sagemath.org/?z=eJwryMkv0Yg21DGM1QQAFu8DVw==&lang=octave

Using my own server plotting in R gives me nothing
(not even "Error in png() : X11 is not available")
and Octave still gives me only Ascii.

I have two related questions:
1. Is it possible to get R plots in sagecell?

It should be.   http://www.dan-erik.com/phdblog/2013/05/using-sage-with-r/ might be helpful.  

But http://statisfactions.com/2013/interactive-r-code-widget/ used to work and doesn't any more, so that doesn't bode well.  Andrey, did you maybe use a version of Sage compiled without png support for R?  Unfortunately even the comment on that page no longer works.

The graphics work with
pdf("Figname.pdf")
plot(...)
dev.off()

[William Stein]

Requiring the user to actually do that is a disaster regarding
usability. In SageMathCloud I just make it automatic. E.g.,doing
this just works:

%r
cars <- c(1, 3, 6, 4, 9)
plot(cars, type="o", col="blue")
title(main="Autos", col.main="red", font.main=4)

See:

https://cloud.sagemath.com/projects/4a5f0542-5873-4eed-a85c-a18c706e8bcd/files/support/2015-10-26-092322-r-plot.sagews

I don't think I've done the analogous thing for Octave yet, but should.

The way it works is by making %r *slower* when used interactively, by
having it check to see if a new graphic was created each time it runs,
and if so, displaying it. The codes all open source. Sage Cell
server should be changed to do the same, in my opinion. However,
leave the sage library's r.eval itself alone, since doing the above
would slow it down.

William


--
William (http://wstein.org)

[Andrey Novoseltsev]

 Hi William,

Can you please give a pointer to where this is implemented in SMC?

Somewhat related: one of the problems with cells is that R works from a wrong directory (home) and writes its files there. They should be instead in a proper temp folder where at least links to plots will be given.

Andrey

[William Stein]

> Can you please give a pointer to where this is implemented in SMC?

This link:

https://github.com/sagemathinc/smc/blob/master/salvus/sage_salvus.py#L1942

but this will become invalid in a few days with a major refactoring of
the code. In any case look for the file sage_salvus.py and a line
that looks like this:

# Monkey patch the R interpreter interface to support graphics, when
used as a decorator.

It's like one page of code total.

> Somewhat related: one of the problems with cells is that R works from a
> wrong directory (home) and writes its files there. They should be instead in
> a proper temp folder where at least links to plots will be given.
>
> Andrey
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-support...@googlegroups.com.
> To post to this group, send email to sage-s...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-support.
> For more options, visit https://groups.google.com/d/optout.



--
William (http://wstein.org)

[Benedikt Magnússon]

Thanks for the replies.

kcrisman: 

I already installed xorg-dev, as sugested at dan-erik's blog. 

Using dev.off() gives me "null device     1"

Andrey:

Do you think it is possible to use the sage_salvus.py code in Sage Cell?

One more question, is there some log file for sage cell which I dig into?

-Benedikt

[Andrey Novoseltsev]

On Thursday, 29 October 2015 04:12:36 UTC-6, Benedikt Magnússon wrote:

Thanks for the replies.

kcrisman: 

I already installed xorg-dev, as sugested at dan-erik's blog. 

Using dev.off() gives me "null device     1"

My understanding is that this is normal R output and the plot is created. You just don't see it because it is in a wrong directory.
 

Andrey:

Do you think it is possible to use the sage_salvus.py code in Sage Cell?

Yes, I am looking into it. Definitely will not start working today, but perhaps in a week or so I can fix it so that you get a link to "general files" created from R and plots are displayed automatically.

One more question, is there some log file for sage cell which I dig into?

See
https://github.com/sagemath/sagecell/blob/master/log.py
which works in default configuration (I guess messages will just go to your syslog), as well as
https://github.com/sagemath/sagecell/blob/master/contrib/vm/container_manager.py#L171
and
https://github.com/sagemath/sagecell/blob/master/contrib/vm/compute_node/etc/rsyslog.d/sagecell.conf
which are used "in production": there are detailed logs inside of each container and a summary one on the host.

I doubt these logs are very useful with debugging R (or Sage) issues since from the point of view of the cell server "everything works", you just don't have access to results.

[Andrey Novoseltsev]

OK, here is my take on R plots:
http://sagecell.sagemath.org/?z=eJwryMkv0SjKyy_K1TA0MNDU5OUqwBDJKkhN1yjPTCnJsDUGimBTUpCSpoFdAkMkOTGzKD8epCE_LzUtMyfV1s3RJ9iVSN3FZekaIE15ibmptkrOpcUl-bl6QEElrKoB67NCZQ==&lang=r

SageMathCell now defaults to producing SVG plots in R and displays all automatic plots that it can (BMP, JPEG, PNG, SVG). Users are free to start new plotting devices, pass any options to them, and create multiple plots in the same cell. Plots with custom names or harder formats like PDF will be provided as links only. With SageMathCloud approach you cannot change format and changing options is done not from R code using R commands, you are also limited to a single plot displayed automatically. The code above does not show any plots at all in the cloud.

What had to be done:
1) R interface is recreated after each forking (via "r=R()" command) to make sure that a new working directory is picked up. Would be great if interfaces followed changing directories of the main process, but I don't know how feasible that is.
2) There is site configuration file installed into R in Sage setting two options including default graphing device SVG. In Sage R interface is prohibited from picking up configuration from anywhere and device is forced in addition - I had to change it and I think it makes a lot of sense to change it for Sage itself.
3) After each R cell is executed, all graphic devices are now closed and plots with default R names are shown automatically if possible. (Everything else gets a link to.) R plots produced not in "R cells" will be always linked to only, without auto display - this is similar to cloud showing plots only if r-decorator was used.

Andrey

On Monday, 26 October 2015 10:27:23 UTC-6, William wrote:

[kcrisman]

Impressive!

[William Stein]

Thanks -- I've made a ticket:

https://github.com/sagemathinc/smc/issues/234

Where's a link to exactly your code (so I can add it to the SMC ticket)?

[Andrey Novoseltsev]

On Thursday, 5 November 2015 21:16:03 UTC-7, William wrote:

Thanks -- I've made a ticket:

   https://github.com/sagemathinc/smc/issues/234

Where's a link to exactly your code (so I can add it to the SMC ticket)?

Commits linked at github ticket above!

[Andrey Novoseltsev]

On Monday, 26 October 2015 06:13:08 UTC-6, Benedikt Magnússon wrote:

Hi

I wish to be able to plot in Octave/R inside a sagecell.

and in Octave you only get a Ascii plot
https://sagecell.sagemath.org/?z=eJwryMkv0Yg21DGM1QQAFu8DVw==&lang=octave

I have to say that R documentation is more pleasant/clear than Octave/gnuplot.

My understanding is that (unlike for R) we do not try to do anything for Octave plotting, so everything is "default". Plotting commands in Octave use either gnuplot or OpenGL, but it seems that in our setup only gnuplot is available. How extractly it is used I don't know, but my guess is that Octave creates some input file and then passes it to "gnuplot" and you have an option to change the name of the called binary and supply some options to it. For whatever reason, default options result in just ASCII plot in SageMathCell and SageNB. When I try to add explicit commands for saving in a particular format I am just getting the same ASCII plot one more time and some warnings. Starting Octave from a local terminal allows me to open plots in a new window, while saving files works but still shows warning about missing things. It would be nice if someone who knows Octave and gnuplot figured out what do we need to do to get some plots and no warnings - I am stopping for now.

Andrey

[Nils Bruin]

On Friday, November 6, 2015 at 2:59:56 PM UTC-8, Andrey Novoseltsev wrote:

My understanding is that (unlike for R) we do not try to do anything for Octave plotting, so everything is "default". Plotting commands in Octave use either gnuplot or OpenGL, but it seems that in our setup only gnuplot is available. How extractly it is used I don't know, but my guess is that Octave creates some input file and then passes it to "gnuplot" and you have an option to change the name of the called binary and supply some options to it. For whatever reason, default options result in just ASCII plot in SageMathCell and SageNB. When I try to add explicit commands for saving in a particular format I am just getting the same ASCII plot one more time and some warnings. Starting Octave from a local terminal allows me to open plots in a new window, while saving files works but still shows warning about missing things. It would be nice if someone who knows Octave and gnuplot figured out what do we need to do to get some plots and no warnings - I am stopping for now.

I don't know either very much, but I have succeeded on my local octave (on Fedora) to produce some plots in files via essentially this code:

https://sagecell.sagemath.org/?z=eJxLy0wvLUrVMNRRKssszkzKSVXSUcpPS1PS5OUqLi1K0yhITcwuBnESy1ITi0HqwCJ6BXnpIDUAiBwTdA==&lang=octave

figure(1,"visible","off")
surf(peaks)
saveas(1,"peaks.png")

(saving to peak.pdf works too).

I did install "transfig" (a part of xfig), as well as "pstoedit".

Turning the "visible" off ensures that you don't get the silly ascii art.

Other remarks:

I do need to run octave with "DISPLAY" unset. Otherwise it seems to want to use the graphics windows things, and from there any attempt to saving the graphics leads to a segfault [which has significantly diminished my trust in octave, or at least its fedora packaging]. Even with "visible" off, I wasn't getting saved graphics (but no segfaults either).

It would be great if we could get basic octave graphics support running on sagecell. I think plenty of matlab-centric courses could make excellent use of it.

[William Stein]

A basic test case -- which fails horribly in SageMathCloud (in both
worksheets *and* the command line) :

sage: octave.eval("x = -10:0.1:10;\nplot (x, sin (x));")

So thanks for posting your suggestions, which help a lot. SMC Ticket:

https://github.com/sagemathinc/smc/issues/238

William