Re: VirtualBox

[William Stein]

On Mon, Jan 9, 2012 at 7:32 AM, Volker Braun <vbrau...@gmail.com> wrote:
> Hi Wiliam,
>
> Karl-Dieter told me that you found some problem with the virtual
> machine but he didn't recall any details. What exactly is the issue?

Volker,

My perspective changed a lot because I spent so much time during the
last week trying to help many mathematicians to install and use Sage
at a huge conference, the Sage exhibit booth, and the short course.
Here's what I think:

(1) Virtual machines (or possibly co-Linux, maybe) is the only thing
we should put any effort toward for Windows. A native Microsoft C++
port or a Cygwin port are both a waste of time. I thus strongly
applaud your effort to make a VM (and anybody else's).

(2) The virtual machine should be as usable as possible, even if
networking between the VM and guest can't be configured. This means a
GUI, tools (for mouse integration), and at least a web browser are all
installed.

(3) The networking configuration in Linux (inside the VM) needs to be
very robust.

I realize that some of this is philosophically different than what you
think about how a VM should be made for Sage, so I'll do my best to
justify my opinions below.

DETAILS:

(1) I've *used* Sage on Cygwin and worked a lot on the port several
times. The basic problems are that (a) Cygwin is 32-bit only, (b)
forking in windows is massively fubar'd and very slow, (c) DLL
"rebasing" makes Sage-on-cygwin brittle and insane. A massive problem
with a native port is that even if it were done (a big if), it would
be far, far too much work to maintain as new versions of components of
Sage are constantly updated and released, Windows is updated, etc.; we
can't even keep up with OS X releases, where every component is well
supported.

(2) I can't tell you how many times during the last week that I
watched somebody load the Sage VM, see a textbox telling them to
browse to localhost, try to copy it and have their cursor get stuck in
the VM, think their computer crashed, etc., then type it in by hand
somewhere and mess up (e.g., type https instead). Then once they do
all that it often doesn't work anyways. Windows networking on a
laptop is a random beast, often firewalled and fubar'd by mandatory
antivirus software, and half broken. If people have downloaded and
installed Sage, let's at least give them something that immediately
works with 99% probability: have the VM boot up with Firefox
pre-logged into a running local notebook server and mouse integration
so the mouse doesn't get stuck and copy-paste works. It's
definitely possible to do this and not use a lot of disk space by
running from a compressed filesystem (with an overlay so you still
have read/write -- I've done it before). I remember doing this with
only about 600MB of disk space used *after* extracting the zip.

(3) Often when I investigated further I would find that eth0 had
turned into eth1 on bootup, which completely kills everything from the
user's point of view (though the user doesn't even get an error
message!). Nearly anybody using MS Windows is going to be completely
stuck when the solution is: "configure the eth1 interface, run the
notebook command manually from the command line, and memorize a
random-looking 4-digit ip address". I've made VM's for Sage for
years (before I ran out of steam around a year ago), and I remember
spending a lot of time dealing via Python scripts with the possibility
that Linux renames eth0 to eth1, etc. It seems the current VM
doesn't successfully do this.

Anyway, I'm just reporting on what I saw last week. I know that
making VM's is really difficult, can take a huge amount of time, and
testing them is even harder yet. But I'm all for it, since I think a
good VM has the best chance of addressing the "Sage-for-Windows"
problem. However, I think that the solutions offered so far (by me,
you, and others) for a Sage VM on Windows have simply failed to solve
the problem.

I think co-Linux is also worth looking at again. It's cool because it
is *not* a virtual machine (you can even install Windows in
VirtualBox, and install co-Linux into that Windows install). It has
direct access to the windows filesystem, the network, etc., just like
a normal application. But it's Linux. I just checked their website
(http://www.colinux.org/), and they made a release a few months ago,
and also claim to have new corporate sponsorship for a 64-bit port.

-- William

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[Dima Pasechnik]

On Tuesday, 10 January 2012 01:11:37 UTC+8, William wrote:

On Mon, Jan 9, 2012 at 7:32 AM, Volker Braun <vbrau...@gmail.com> wrote:
> Hi Wiliam,
>
> Karl-Dieter told me that you found some problem with the virtual
> machine but he didn't recall any details. What exactly is the issue?

Volker,

My perspective changed a lot because I spent so much time during the
last week trying to help many mathematicians to install and use Sage
at a huge conference, the Sage exhibit booth, and the short course.
Here's what I think:

 (1) Virtual machines (or possibly co-Linux, maybe) is the only thing
we should put any effort toward for Windows.  A native Microsoft C++
port or a Cygwin port are both a  waste of time.   I thus strongly
applaud your effort to make a VM (and anybody else's).

Yes, I think it's a good idea to stop spending time etc on Cygwin port.

It is a huge PITA to use it in its present state (having spent quite a bit of time on it last year) , 

and unless there is a drastic improvement coming in (e.g., a good 64-bit port, which would actually fix the fork blues, AFAIK),  it's not going to work well, ever.

coLinux looks promising. What does stop one from putting Sage on it presently?

Dima

[kcrisman]

On Jan 9, 1:18 pm, Dima Pasechnik <dimp...@gmail.com> wrote:
> On Tuesday, 10 January 2012 01:11:37 UTC+8, William wrote:
>
> > On Mon, Jan 9, 2012 at 7:32 AM, Volker Braun <vbrau...@gmail.com> wrote:
> > > Hi Wiliam,
>
> > > Karl-Dieter told me that you found some problem with the virtual
> > > machine but he didn't recall any details. What exactly is the issue?
>
> > Volker,
>
> > My perspective changed a lot because I spent so much time during the
> > last week trying to help many mathematicians to install and use Sage
> > at a huge conference, the Sage exhibit booth, and the short course.
> > Here's what I think:
>
> >  (1) Virtual machines (or possibly co-Linux, maybe) is the only thing
> > we should put any effort toward for Windows.  A native Microsoft C++
> > port or a Cygwin port are both a  waste of time.   I thus strongly
> > applaud your effort to make a VM (and anybody else's).
>
> Yes, I think it's a good idea to stop spending time etc on Cygwin port.
> It is a huge PITA to use it in its present state (having spent quite a bit
> of time on it last year) ,

True, though it's frustrating to have to count that all as a sunk
cost :(

What about Emil's LiveCD and bootable partition thingie solutions as
well? Certainly the one-click version is a possible solution as well,
if it's robust.

[Volker Braun]

I had looked at coLinux before settling on VirtualBox. Since coLinux
also needs a low-level driver I don't see any compelling advantage for
our use. It is harder to install than VirtualBox and you can't run X
on top of it right now. VirtualBox will be slightly slower in the
execution, but its less error prone imho.

[William Stein]

> coLinux looks promising. What does stop one from putting Sage on it
> presently?
>
> Dima

Nothing. I've done it before several times. I was hoping with my
email to encourage you (meaning anybody reading this!) to try it.

What stops me right now is that I don't have Windows installed
anywhere at all.

-- William

[William Stein]

On Mon, Jan 9, 2012 at 10:41 AM, kcrisman <kcri...@gmail.com> wrote:
>> Yes, I think it's a good idea to stop spending time etc on Cygwin port.
>> It is a huge PITA to use it in its present state (having spent quite a bit
>> of time on it last year) ,
>
> True, though it's frustrating to have to count that all as a sunk
> cost :(

I know. I assure you, I've "wasted" even more time on Sage + Windows
than you.

> What about Emil's LiveCD and bootable partition thingie solutions as
> well?  Certainly the one-click version is a possible solution as well,
> if it's robust.

I have never "believed" in the viability of live CD's for much beyond
demos, sysadmin work, and hacking into computers when you have
hardware access. I don't think live CD's are in any way a viable
solution for Sage on Windows. It just doesn't make sense at all
for "daily users" -- do you really expect them to drop everything
(stop checking Facebook, their email, stop using their favorite
editors, etc.) to reboot into Linux *just* to run Sage? Sure, that
would be great for a workshop, but for daily use it is completely and
totally ludicrous. So I applaud and appreciate the work to create
LiveCD's, and I see them as potentially very valuable for workshops
and certain other applications, but they are definitely no strategy
for adoption of Sage on Windows.

William

[William Stein]

On Mon, Jan 9, 2012 at 10:48 AM, Volker Braun <vbrau...@gmail.com> wrote:
> I had looked at coLinux before settling on VirtualBox. Since coLinux
> also needs a low-level driver I don't see any compelling advantage for
> our use. It is harder to install than VirtualBox and

I have the impression that you didn't seriously use coLinux. It's a
port of the Linux kernel to run as a native Windows application. It's
different than virtual machines like VirtualBox or VMware.

There might be compelling advantages:

* direct access to the native Windows filesystem -- no weird stuff
with networking or VM shared folders needed.

* networking is different in some good ways (and some confusing ways).

Being a genuine application has its advantages.

> you can't run X on top of it right now.

This http://www.andlinux.org/ is a specific install of colinux that
some people put together for their company use, and give away free.
It includes X. So maybe the situation is more complicated.

We used to have a Sage distribution based on andLinux, but the guy
working on it died.

> VirtualBox will be slightly slower in the execution,

I found the opposite in practice. I did benchmarks when we had made
available both 32-bit VMware and coLinux versions of Sage on Windows.
In fact, to my surprise, coLinux was significantly slower for raw
computation than VMware. I remember it being about 60-70% the speed
of VMware running Linux with Sage installed. I don't know why.
Because of VTX extensions, VMware and VirtualBox are amazingly fast
for raw CPU-bound computation.

> but its less error prone imho.

This is probably true. VirtualBox is massively more widely used.
When we used to distribute a coLinux version of Sage (in 2006), I
remember watching many Windows laptops crash hard due to their
low-level driver. But that was 5 years ago, and things may have
changed by now.

-- William

[mmarco]

So, summarizing, a (let's say) VirtualBox VM, that boots directly into
simple X environment with sage running on the background+firefox open
pointing at it would be a good solution?

That seems feasible. The bigest problem i see is the mouse
integration.

What about a VM with no X that just boots, automatically starts sage
and shows a message "open your browser and go to http://localhost:8000"
(the port can be forwarded in the VM configuration)??. Would that give
problems depending on the status of the windows network?

[Volker Braun]

On Monday, January 9, 2012 3:25:54 PM UTC-5, mmarco wrote:

That seems feasible. The bigest problem i see is the mouse
integration.

Whats the problem with mouse integration? We just have to make sure that the guest extensions are loaded in VirtualBox, then you can move the mouse pointer seamlessly beween the guest and host OS.

[mmarco]

Then its even better that i thought. I always tried VirtualBox without
the extensions, so the mouse integration is a problem.

[Volker Braun]

Its true that I haven't spent much time with coLinux, I just read in the FAQ that one can't run an X server natively.  http://www.andlinux.org runs Xming (a Windows X server) that coLinux connects to through the network. So you need to configure your firewall to allow tcp port 6000. In fact, the coLinux FAQ #1 is: "Please ensure that your firewall does not block traffic on the 'TAP-Colinux' interface! Furthermore, the executables Xming.exe and pulseaudio.exe are often blocked by software firewalls."

[Georg S. Weber]

>
> > but its less error prone imho.
>
> This is probably true.  VirtualBox is massively more widely used.
> When we used to distribute a coLinux version of Sage (in 2006), I
> remember watching many Windows laptops crash hard due to their
> low-level driver.   But that was 5 years ago, and things may have
> changed by now.
>
>  -- William

With a well-maintained "Sage deployment via VirtualBox image"
solution, we would not only address the Windows OS(es)--- but every
system where (the minimal version required by the featuers Sage uses
of) VirtualBox runs. Both from the user *and* the developer point of
view (I easily can load a certain VB image on my OS X and check some
error report)!

And especially, whenever a new Fedora, RedHat, Ubuntu, Debian,
OpenSuse, OS X, Solaris, Windows ... version comes out, where we
didn't have the time/resources/whatever yet to make Sage work on it
"natively", we always could tell the users at least to use that "Just
works!" solution as a fallback.

Since there seems to be some need in discussing details, making some
decisions, and (most importantly!) to document the latter, why not use
the http://wiki.sagemath.org/devel/SEP process (or is there an
alternative, better place to do "requirements fixation" for such Sage
develop tasks)?
Which reminds to check whether I still remember my Sage Wiki login and
password ...


Cheers,
Georg

[William Stein]

On Mon, Jan 9, 2012 at 12:54 PM, Georg S. Weber
<georg...@googlemail.com> wrote:
>>
>> > but its less error prone imho.
>>
>> This is probably true.  VirtualBox is massively more widely used.
>> When we used to distribute a coLinux version of Sage (in 2006), I
>> remember watching many Windows laptops crash hard due to their
>> low-level driver.   But that was 5 years ago, and things may have
>> changed by now.
>>
>>  -- William
>
> With a well-maintained "Sage deployment via VirtualBox image"
> solution, we would not only address the Windows OS(es)--- but every
> system where (the minimal version required by the featuers Sage uses
> of) VirtualBox runs. Both from the user *and* the developer point of
> view (I easily can load a certain VB image on my OS X and check some
> error report)!
>
> And especially, whenever a new Fedora, RedHat, Ubuntu, Debian,
> OpenSuse, OS X, Solaris, Windows ... version comes out, where we
> didn't have the time/resources/whatever yet to make Sage work on it
> "natively", we always could tell the users at least to use that "Just
> works!" solution as a fallback.

Let's focus our energy on doing this. Let's also plan to fully
support having both a 32 and 64-bit version. Right now I think we
only provide a 32-bit VM. If we are organized, we can do both 32 and
64-bit.

william

>
> Since there seems to be some need in discussing details, making some
> decisions, and (most importantly!) to document the latter, why not use
> the http://wiki.sagemath.org/devel/SEP process (or is there an
> alternative, better place to do "requirements fixation" for such Sage
> develop tasks)?
> Which reminds to check whether I still remember my Sage Wiki login and
> password ...
>
>
> Cheers,
> Georg
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

[William Stein]

You just described what the VM Volker made already does. This is most
certainly not sufficient for many users in real life situations, e.g.,
where networking doesn't work.

By the way, how can http://localhost:8000 ever work? If I'm using
VirtualBox and I go to http://localhost:8000 in Internet Explorer, I'm
definitely *not* going to get something having anything to do with the
virtual machine.

>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--

[Volker Braun]

On Monday, January 9, 2012 4:07:49 PM UTC-5, William wrote:

Let's focus our energy on doing this.  

Agreed!

Let's also plan to fully
support having both a 32 and 64-bit version.  Right now I think we
only provide a 32-bit VM.  If we are organized, we can do both 32 and
64-bit.

I don't see any compelling reason for  a 64bit guest. Many people don't know whether they run a 32 or 64bit host. VirtualBox lets you pretty much mix&match guest and host (including 64bit guest on 32bit host) but especially the latter requires hardware virtualization capabilities in the CPU. Its easier to just have everybody run 32bit guests. We can build a 64bit guest, but in any case we should point everybody to the 32bit version.

[mmarco]

> By the way, how canhttp://localhost:8000ever work? If I'm using
> VirtualBox and I go tohttp://localhost:8000in Internet Explorer, I'm

> definitely *not* going to get something having anything to do with the
> virtual machine.
>

You can configure port forwarding with VirtualBox. If you forward port
8000 of the host system to port 8000 of the guest system, you get that
result.

[Julien Puydt]

Le 09/01/2012 22:09, William Stein a �crit :

> By the way, how can http://localhost:8000 ever work? If I'm using
> VirtualBox and I go to http://localhost:8000 in Internet Explorer, I'm
> definitely *not* going to get something having anything to do with the
> virtual machine.

The browser could be _in_ the VM ; that would make it easy to ensure
that both http://localhost:8000 and jsmath work.

Snark on #sagemath

[Volker Braun]

On Monday, January 9, 2012 4:19:36 PM UTC-5, mmarco wrote:

You can configure port forwarding with VirtualBox. If you forward port
8000 of the host system to port 8000 of the guest system, you get that
result.

Thats what my virtual machine does, but let's face it: Networking is pretty much foobared on most windows machines. All the different AV vendors doing different stupid things to firewall off ports are a support nightmare. This was always one of the drawbacks of my attempt, though I am hoping that things are getting better over time here.  

[mmarco]

Is there another port that is usually not firewalled that could be
used?

[Julien Puydt]

Le 09/01/2012 22:26, Julien Puydt a �crit :

> The browser could be _in_ the VM ; that would make it easy to ensure
> that both http://localhost:8000 and jsmath work.

... and no firewall & AV problems either ;-)

Snark on #sagemath

[Keshav Kini]

> Which reminds to check whether I still remember my Sage Wiki login and
> password ...

I'll just jump in to say that a little while ago we connected the Sage Wiki logins to the trac user store, so you should log into the Sage Wiki using your trac user/pass. Though your email was five hours ago so maybe you have figured this out already...

-Keshav

----
Join us in #sagemath on irc.freenode.net !

[William Stein]

On Mon, Jan 9, 2012 at 1:16 PM, Volker Braun <vbrau...@gmail.com> wrote:
> On Monday, January 9, 2012 4:07:49 PM UTC-5, William wrote:
>>
>> Let's focus our energy on doing this.
>
> Agreed!
>
>> Let's also plan to fully
>> support having both a 32 and 64-bit version.  Right now I think we
>> only provide a 32-bit VM.  If we are organized, we can do both 32 and
>> 64-bit.
>
>
> I don't see any compelling reason for  a 64bit guest.

I immediately see at least 3 compelling reasons for 64-bit:

1) 64 bit is much, much faster in some cases, e.g., MPIR gets huge
performance gains, which impacts everything else I care about.
Moreover, it is also much faster for doing arithmetic with machine
longs, which are 64-bit, which makes certain computations faster.

2) One can use much more RAM with 64-bit. Many computers these days
have 8GB RAM (e.g., even my laptop). Again, at least for many
computations I do, RAM is the critical bottleneck.

3) There is some interesting software that is 64-bit only, e.g., psage
includes an important program called "smalljac" by Drew Sutherland
that is 64-bit only.

> Many people don't know whether they run a 32 or 64bit host.

True, but when discussing "64-bit" we should broaden our horizon to
consider that there are Windows users who are very technically
sophisticated.

> VirtualBox lets you pretty much
> mix&match guest and host (including 64bit guest on 32bit host) but
> especially the latter requires hardware virtualization capabilities in the
> CPU. Its easier to just have everybody run 32bit guests.

I can't argue with you that it is easier.

> We can build a
> 64bit guest, but in any case we should point everybody to the 32bit version.

I think the Ubuntu download page is a good model for 32 versus 64-bit
downloads. Check it out:

http://www.ubuntu.com/download/ubuntu/download

-- William

[William Stein]

Thanks for the explanation; that's very cool. I assume this is what
Volker's VM does by default?

I wonder how VirtualBox deals with two hosts that both have the same
port forwarded...

-- William

[Volker Braun]

On Monday, January 9, 2012 11:14:56 PM UTC-5, William wrote:

> You can configure port forwarding with VirtualBox. If you forward port
> 8000 of the host system to port 8000 of the guest system, you get that
> result.

Thanks for the explanation; that's very cool.  I assume this is what
Volker's VM does by default?

Yes, and also host:2222 to guest:22 so you can ssh into the guest.

 

I wonder how VirtualBox deals with two hosts that both have the same
port forwarded...

 

Not at all; Only one process(*) can bind to a port and wait for connections. First one wins. Unless some craptastic software firewall disallows it altogether, then everybody loses out.

(*) with exceptions on some systems

[Volker Braun]

On Monday, January 9, 2012 11:13:27 PM UTC-5, William wrote:

2) One can use much more RAM with 64-bit.  Many computers these days
have 8GB RAM (e.g., even my laptop).

Or 16GB in my laptop ;-)  Though its really only sensible to use ~50% for the VM, which brings us back to the 4GB limit on a 8GB machine.

The 64-bit virtual machine should then also come with more ram & cpus configured, and require hardware virtualisation. In other words, be only suitable for a modern computer.

[William Stein]

On Mon, Jan 9, 2012 at 8:35 PM, Volker Braun <vbrau...@gmail.com> wrote:
> On Monday, January 9, 2012 11:13:27 PM UTC-5, William wrote:
>>
>> 2) One can use much more RAM with 64-bit.  Many computers these days
>> have 8GB RAM (e.g., even my laptop).
>
> Or 16GB in my laptop ;-)  Though its really only sensible to use ~50% for
> the VM, which brings us back to the 4GB limit on a 8GB machine.

Of course, memory is just one of the advantages. That said, I do most
of my work on my laptop under Linux using VirtualBox, and I set the
limit to "5719MB", which is in the green still (in the VirtualBox
GUI).

> The 64-bit virtual machine should then also come with more ram & cpus
> configured, and require hardware virtualisation. In other words, be only
> suitable for a modern computer.

+1 It's what I would use if I had to use Windows.

[Jason Grout]

On 1/9/12 11:42 PM, William Stein wrote:
> +1 It's what I would use if I had to use Windows.

You do it even on a mac!

Jason

[Dima Pasechnik]

On Tuesday, 10 January 2012 03:06:14 UTC+8, William wrote:

> coLinux looks promising. What does stop one from putting Sage on it
> presently?
>
> Dima

Nothing.  I've done it before several times.  I was hoping with my
email to encourage you (meaning anybody reading this!) to try it.

on a 64-bit Windows 7 I have, I cannot install coLinux: 

it aborts with a message saying it cannot run on a 64-bit system.

(and this is also according to Q27 in http://colinux.wikia.com/wiki/FAQ)

[Florent Hivert]

Hi There,

On Mon, Jan 09, 2012 at 06:42:16PM -0800, Keshav Kini wrote:
> > Which reminds to check whether I still remember my Sage Wiki login and
> > password ...
>
> I'll just jump in to say that a little while ago we connected the Sage Wiki
> logins to the trac user store, so you should log into the Sage Wiki using
> your trac user/pass. Though your email was five hours ago so maybe you have
> figured this out already...

Speaking about the wiki, I already posted here messages asking for help with
the connection on the wiki. Here is the problem:

When I click to connection I'm asked for a login password. I then enter my
trac account infos. I'm now shown a page starting by
* hivert * Settings * Logout
Now any further click, either <Edit> or a hyperlink in the page simply log me
out. As a consequence, I'm not able to edit any page.

The problem arose with firefox, konqueror and opera on different machines with
different accounts. So I really think it is on the server side.

Any help appreciated, especially if I'm not asking at the correct place I'd
like to know it,

Cheers,

Florent

[mmarco]

I have no contact with windows systems at all, so i don't know how is
the real situation right now.

Is it usual that windows systems come with port 8000 firewalled? How
hard would it be for an average user to open that port?

And also, does the port forwarding method work even if there is no
network connection in the host system?

[mmarco]

Oh, and by the way: what about the option of running the VM in
headless mode? That would definitely solve the problem of the cursor
stuck that William pointed. And would make easyer to do a simple "one
click launch" script.

[Volker Braun]

On Tuesday, January 10, 2012 8:22:51 AM UTC-5, mmarco wrote:

Oh, and by the way: what about the option of running the VM in
headless mode?

Again, that is what my current virtual machine does. It works fine as long as the host has networking up and running. 

Its usual, though, for Windows machines to come with antivirus (or, perhaps even more likely, virus) installs of questionable qualitiy that mess up things. 

[RegB]

As a very much below average user I will try that right now.
Windows 7, new Laptop, previous laptop was Vista based, so this could
take a bit of time.
Now 9:16
Going into Windows control panel first;
Troubleshooting>Network & Internet>Incoming connections :::: gets to
some sort of automated troubleshooter script, nothing asking me what I
am trying to do, no clues about what is being blocked or where.
Interesting... If I click on the advanced button it asks if I want to
run as administrator, which I then clicked and one of the network
adapters that it offered to diagnose is "Virtual Box Host-Only
Network".
OK, that didn't find a problem, but of course there isn't one, I am
looking for a way through a block that the troubleshooter probably
regards as "good".
I got a network troubleshooting log file, it has details of the
connection as a header, then all the usual MS nonsense about trying a
different cable, switching things off, waiting 30 seconds, switching
back on, etc.
I think this is going nowhere.

Ahh, Windows Firewall>Allowed Programs :::brings me to a window that
offers
"Allow another Program" in the bottom right corner, I clicked on that
and selected "Oracle VM Virtual Box" which then appeared in the table
with local checked and public unchecked.
If this is all I need - now 9:43 (with commentary 27 minutes).

This might be useful to me, it might even get the shared folders thang
to work (-:
As I said, this is a NEW laptop and I am new to Windoze7.
I shall now go into my virtual machines to see if they can see the
host's shared folders.

FWIW, etc.

[mmarco]

>
> Again, that is what my current virtual machine does. It works fine as long
> as the host has networking up and running.

So port forwarding in windows does not work if the host system does
not have a network connection? That really complicates things.

That would leave the aproach of shipping a full system with sage+X
server+browser included as a better option. the problem i would see
two problems with that:

1) The notework files are "isolated" inside the VM. If there is no web
access from the host, it can be difficult to, for example, download
a .sws from the web and import it to the local copy of sage. Or vice-
versa: export a local worksheet to a file that can be sent to somebody
else. Maybe this issue could be somehow circumvented by sharing
folders between host and guest, but then again, maybe this would need
a working network in the host. It would also need to have the guest
additions installed. This guest aditions are distributed under a
propietary license, but luckyly it includes an express permision for
distributing VM's with them installed.

2) The mouse integration issue. It seems that the guest aditions would
also solve this problem.

So i guess the "full batteries included" aproach is more convenient
here.

[William Stein]

On Jan 10, 2012 12:50 AM, "Dima Pasechnik" <dim...@gmail.com> wrote:
>
>
>
> On Tuesday, 10 January 2012 03:06:14 UTC+8, William wrote:
>>
>> > coLinux looks promising. What does stop one from putting Sage on it
>> > presently?
>> >
>> > Dima
>>
>> Nothing.  I've done it before several times.  I was hoping with my
>> email to encourage you (meaning anybody reading this!) to try it.
>
>
> on a 64-bit Windows 7 I have, I cannot install coLinux: 
> it aborts with a message saying it cannot run on a 64-bit system.
> (and this is also according to Q27 in http://colinux.wikia.com/wiki/FAQ)

Thanks, that is very useful to know, as 64-bit is finally becoming common for Windows, I think.

>
>  
>>
>> What stops me right now is that I don't have Windows installed
>> anywhere at all.
>>
>>  -- William
>

[Volker Braun]

On Tuesday, January 10, 2012 10:15:18 AM UTC-5, mmarco wrote:

> Again, that is what my current virtual machine does. It works fine as long
> as the host has networking up and running.
So port forwarding in windows does not work if the host system does
not have a network connection? That really complicates things.

You don't need an outside network connection, the loopback device is just fine. Unless it is firewalled.

Also, having the VM run X+web browser doesn't prevent us from tunneling the notebook port to the host. So if you want to keep using the host browser thats fine, too.

[mmarco]

Ok, so the main problem with your VM would be the firewalls and
antivirus?

[William Stein]

On Jan 10, 2012 6:37 AM, "Volker Braun" <vbrau...@gmail.com> wrote:
>
> On Tuesday, January 10, 2012 8:22:51 AM UTC-5, mmarco wrote:
>>
>> Oh, and by the way: what about the option of running the VM in
>> headless mode?
>
>
> Again, that is what my current virtual machine does. It works fine as long as the host has networking up and running. 
>

I'm not sure what you guys means by "headless".  I would definitely not call Volker's current VM headless, since when you start it up there is a visible Linux console (which I would call a "head"), and if you click in it, then the cursor is trapped. 

-- William

[mmarco]

What i meant by headless is to run it completelly in the background,
with no window showing at all. You can run a VM in that way with the
VBoxHeadless command.

[Volker Braun]

On Tuesday, January 10, 2012 10:51:15 AM UTC-5, William wrote:

I'm not sure what you guys means by "headless".  I would definitely not call Volker's current VM headless, since when you start it up there is a visible Linux console (which I would call a "head"), and if you click in it, then the cursor is trapped. 

True, but it would be easy to write a batch file than executes

VBoxManage startvm "Sage-4.7.2" --type headless

then no window whatsoever would not show up.

[William Stein]

That is a really interesting idea! We would couple this with a
standard Windows GUI App, just like with the Sage app on OS X. If
the Windows GUI App we wrote decides that things aren't working right
(it will run some tests) based on the network, it will offer some
options, maybe including:

* view a console
* ssh into the machine
* bring up a full GUI inside the machine.
* suggest how to debug network/firewall/antivirus issues

Who here has experience with Windows GUI programming? I used to do a
lot of it back in 1993... but haven't since then!

-- William

[Keshav Kini]

Can we adapt the IPython Qt app? Or did you want something more than that? From what I've heard, the IPython Qt app would work well in this scenario since it can connect over zeromq to the backend inside the virtual machine while itself running natively on Windows.

[William Stein]

On Tue, Jan 10, 2012 at 8:38 AM, Keshav Kini <kesha...@gmail.com> wrote:
> Can we adapt the IPython Qt app? Or did you want something more than that?
> From what I've heard, the IPython Qt app would work well in this scenario
> since it can connect over zeromq to the backend inside the virtual machine
> while itself running natively on Windows.

That's definitely not in any way, shape, or form, what I had in mind.
I meant something that appears to the user very similar as the OS X
Sage App. Since you (Keshav) maybe don't use OS X, you might not know
what that means.

This would be a brand new Windows Application that (1) launches the
headless VBox machine, when the OS X App would launch the command line
Sage notebook server, and (2) pops up a web browser pointed at ones
list of published worksheets.

One other thing that would make this massively better. It would be
cool to modify the notebook somehow so that it could store all the
worksheets on the host Windows filesystem instead of inside the VM.
Right now, when people delete the VM and install a new one, all of
their worksheets simply vanish forever, and they likely shout out
curses at Sage (rightfully so).

William

>
> -Keshav
>
> ----
> Join us in #sagemath on irc.freenode.net
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--

[kcrisman]

On Jan 10, 11:42 am, William Stein <wst...@gmail.com> wrote:

> On Tue, Jan 10, 2012 at 8:38 AM, Keshav Kini <keshav.k...@gmail.com> wrote:
> > Can we adapt the IPython Qt app? Or did you want something more than that?
> > From what I've heard, the IPython Qt app would work well in this scenario
> > since it can connect over zeromq to the backend inside the virtual machine
> > while itself running natively on Windows.
>
> That's definitely not in any way, shape, or form, what I had in mind.
>   I meant something that appears to the user very similar as the OS X
> Sage App.  Since you (Keshav) maybe don't use OS X, you might not know
> what that means.
>
> This would be a brand new Windows Application that (1) launches the
> headless VBox machine, when the OS X App would launch the command line
> Sage notebook server, and (2) pops up a web browser pointed at ones
> list of published worksheets.
>

Could this then (perhaps) work with clicking a .sws file, ala
http://trac.sagemath.org/sage_trac/ticket/8473 ? That would be
*really* cool. Volker intimates this should be possible...

[mmarco]

> One other thing that would make this massively better. It would be
> cool to modify the notebook somehow so that it could store all the
> worksheets on the host Windows filesystem instead of inside the VM.
> Right now, when people delete the VM and install a new one, all of
> their worksheets simply vanish forever, and they likely shout out
> curses at Sage (rightfully so).

Thats a good idea, but how can we do that in such a way that it does
work when the host system does not have a working network?

I can imagine a way to do it, but it is not trivial at all: using the
port forwarding to comunicate the windows GUI app with some daemon
inside the guest that transfers the files on the fly each time the
notebook wants to read or write a file.

[Keshav Kini]

On Wed, Jan 11, 2012 at 00:42, William Stein <wst...@gmail.com> wrote:
> On Tue, Jan 10, 2012 at 8:38 AM, Keshav Kini <kesha...@gmail.com> wrote:
>> Can we adapt the IPython Qt app? Or did you want something more than that?
>> From what I've heard, the IPython Qt app would work well in this scenario
>> since it can connect over zeromq to the backend inside the virtual machine
>> while itself running natively on Windows.
>
> That's definitely not in any way, shape, or form, what I had in mind.
>  I meant something that appears to the user very similar as the OS X
> Sage App.  Since you (Keshav) maybe don't use OS X, you might not know
> what that means.

Indeed, I don't have a Mac and don't really know what the Sage App
does. I guessed it was something like Sage App is to Sage console as
IPython Qt console is to IPython terminal interface.

> One other thing that would make this massively better.  It would be
> cool to modify the notebook somehow so that it could store all the
> worksheets on the host Windows filesystem instead of inside the VM.
> Right now, when people delete the VM and install a new one, all of
> their worksheets simply vanish forever, and they likely shout out
> curses at Sage (rightfully so).

This is easy. Virtualbox allows you to mount external directories
inside a Linux VM - the Virtualbox guest additions provide a
filesystem type called "vboxsf" (virtualbox shared folder) to `mount`,
and device names corresponding to directories on the host's filesystem
(which must be configured in the VM's parameters).

[William Stein]

On Tue, Jan 10, 2012 at 10:48 AM, Keshav Kini <kesha...@gmail.com> wrote:
> On Wed, Jan 11, 2012 at 00:42, William Stein <wst...@gmail.com> wrote:
>> On Tue, Jan 10, 2012 at 8:38 AM, Keshav Kini <kesha...@gmail.com> wrote:
>>> Can we adapt the IPython Qt app? Or did you want something more than that?
>>> From what I've heard, the IPython Qt app would work well in this scenario
>>> since it can connect over zeromq to the backend inside the virtual machine
>>> while itself running natively on Windows.
>>
>> That's definitely not in any way, shape, or form, what I had in mind.
>>  I meant something that appears to the user very similar as the OS X
>> Sage App.  Since you (Keshav) maybe don't use OS X, you might not know
>> what that means.
>
> Indeed, I don't have a Mac and don't really know what the Sage App
> does. I guessed it was something like Sage App is to Sage console as
> IPython Qt console is to IPython terminal interface.

No, absolutely not. That is as wrong a perspective as possible, so if
you still think that then somebody needs to make a screencast.

>
>> One other thing that would make this massively better.  It would be
>> cool to modify the notebook somehow so that it could store all the
>> worksheets on the host Windows filesystem instead of inside the VM.
>> Right now, when people delete the VM and install a new one, all of
>> their worksheets simply vanish forever, and they likely shout out
>> curses at Sage (rightfully so).
>
> This is easy. Virtualbox allows you to mount external directories
> inside a Linux VM - the Virtualbox guest additions provide a
> filesystem type called "vboxsf" (virtualbox shared folder) to `mount`,
> and device names corresponding to directories on the host's filesystem
> (which must be configured in the VM's parameters).

I know about that, but I would not call it "easy". There are subtle
issues, e.g., where on the host's filesystem should Sage store its
files. There are also issues with getting it to actually work in
practice robustly. But it's probably the best approach given how the
notebook currently works.

-- William

>
> -Keshav
>
> ----
> Join us in #sagemath on irc.freenode.net
>

[mmarco]

Well, (very quick and dirty) hack would be to make the GUI app to
sync (through ssh, for example) some folder in the host to the /
home/.sage directory at startup, and then sync back periodically.

Other option would be to let the GUI app manage the shared folders
configuration (again, by VBoxManage).

[Keshav Kini]

> No, absolutely not.  That is as wrong a perspective as possible, so if
> you still think that then somebody needs to make a screencast.

Well, since you said it was "definitely not in any way, shape, or
form, what [you] had in mind", I certainly don't *still* think that,
it's just the wrong preconception I had earlier :) I'd like to see a
screencast, if someone makes one. Or I can find someone with a Mac and
ask them to show it to me at some point. Surprisingly, there doesn't
seem to be much info about the Sage App on the Sage website (or maybe
I just can't find it).

> I know about that, but I would not call it "easy".  There are subtle
> issues, e.g., where on the host's filesystem should Sage store its
> files.   There are also issues with getting it to actually work in
> practice robustly.   But it's probably the best approach given how the
> notebook currently works.

I see... that sucks. I've never used it for any particular strenuous
applications so I guess I haven't seen the robustness issues. As for
the location of the files, at least, I think the "correct" place to
store sagenb stuff on Windows is %APPDATA%\.sagenb\ (essentially
equivalent to $HOME/.sagenb/ on *nix).

-Keshav

----
Join us in #sagemath on irc.freenode.net !

[William Stein]

On Tue, Jan 10, 2012 at 11:17 AM, Keshav Kini <kesha...@gmail.com> wrote:
>> No, absolutely not.  That is as wrong a perspective as possible, so if
>> you still think that then somebody needs to make a screencast.
>
> Well, since you said it was "definitely not in any way, shape, or
> form, what [you] had in mind", I certainly don't *still* think that,
> it's just the wrong preconception I had earlier :) I'd like to see a
> screencast, if someone makes one. Or I can find someone with a Mac and
> ask them to show it to me at some point. Surprisingly, there doesn't
> seem to be much info about the Sage App on the Sage website (or maybe
> I just can't find it).
>
>> I know about that, but I would not call it "easy".  There are subtle
>> issues, e.g., where on the host's filesystem should Sage store its
>> files.   There are also issues with getting it to actually work in
>> practice robustly.   But it's probably the best approach given how the
>> notebook currently works.
>
> I see... that sucks. I've never used it for any particular strenuous
> applications so I guess I haven't seen the robustness issues.

I think it works pretty well once it is setup. My main worry is
setting it up automatically in the first place. We'll only know how
well it works if we try.

> As for
> the location of the files, at least, I think the "correct" place to
> store sagenb stuff on Windows is %APPDATA%\.sagenb\ (essentially
> equivalent to $HOME/.sagenb/ on *nix).

Cool.

>
> -Keshav
>
> ----
> Join us in #sagemath on irc.freenode.net !
>

[Jonathan Bober]

On Tue, Jan 10, 2012 at 7:17 AM, William Stein <wst...@gmail.com> wrote:

On Jan 10, 2012 12:50 AM, "Dima Pasechnik" <dim...@gmail.com> wrote:
>
>
>
> On Tuesday, 10 January 2012 03:06:14 UTC+8, William wrote:
>>
>> > coLinux looks promising. What does stop one from putting Sage on it
>> > presently?
>> >
>> > Dima
>>
>> Nothing.  I've done it before several times.  I was hoping with my
>> email to encourage you (meaning anybody reading this!) to try it.
>
>
> on a 64-bit Windows 7 I have, I cannot install coLinux: 
> it aborts with a message saying it cannot run on a 64-bit system.
> (and this is also according to Q27 in http://colinux.wikia.com/wiki/FAQ)

Thanks, that is very useful to know, as 64-bit is finally becoming common for Windows, I think.

I'm pretty sure that most computers sold to the "general public" today will have 64 bit Windows installed, and this has probably been true for a few years. (Just browse the Best Buy website and you will see that most standard laptops have at least 4 gigs of ram and come with Windows 7 Home Premium 64 bit.) I suspect that new computers will probably only have 32 bit Windows installed either because they are cheap netbooks which come with Windows 7 starter or because someone has specifically requested something 32 bit for legacy compatibility reasons.

I've been puzzled for a while about why Ubuntu encourages the 32 bit version as the recommended download. There used to be some problems with flash on 64 bits, but I think those problems are pretty much taken care of now, and I haven't come across any other software that I want to run that has problems with 64 bit linux. (Not that I really _want_ to run flash.)

So I think that there should be a 64 bit version, and we should encourage anyone who can use a 64 bit version to use the 64 bit version, and we should look forward to the day, possibly only a few years away, when Sage can drop support for 32 bit operating systems. (Though maybe this day is more than a few years away, since ARM is lagging behind in adopting 64 bit, and more and more people may be interested in running Sage on ARM as time goes on.)

[mmarco]

There is another consideration to be taken into account: if we
distribute a VM with the virtualbox guest additions installed, we can
distribute it... but i am not sure under which license. It could be
that the user should accept the VirtualBox PUEL license, and, for
example, it is not clear if the users that download it would have the
right to distribite it again. As a curious fact, the license states
that it is not licensed for use in nuclear facilities. Maybe we should
contact Oracle and ask them about it.

[Volker Braun]

The only thing that is under the PUEL is the extension pack (most notably USB support). The main VirtualBox and the shared folder and mouse pointer extensions are under the terms of the GPLv2.

[Volker Braun]

Here is a new version of the virtual machine that now opens up a browser inside the VM in addition to forwarding port 8000 with the host. So a novice user can just fire it up and it boots right into the notebook. More advanced users can use the host browser if they prefer.

http://boxen.math.washington.edu/home/vbraun/sage-4.8.alpha6.ova

If you use windows or OSX then please give it a try and post your experience!

[mmarco]

This really looks like a great out-of-the-box solution. But it takes
around a minute and a half since i start the VM till i get the working
notebook. Can this startup time be improved?

[Volker Braun]

Instead of shutting down the virtual machine you can just save the machine state. Loading the vm then circumvents the whole boot process.

[Benjamin Jones]

On Tue, Jan 10, 2012 at 7:48 PM, Volker Braun <vbrau...@gmail.com> wrote:
> Instead of shutting down the virtual machine you can just save the machine
> state. Loading the vm then circumvents the whole boot process.
>

It looks like it works great on Mac OS X. The VM boots, and right away
the browser loads full screen logged into the notebook.

There are some testing worksheets there (says Volker) and the Java
runtime is missing (Volker is aware of this also). I was wondering a
couple of things about the usability of Sage in the VM browser (and
bear in mind that I haven't used the VM appliance at all before).

After a minute or two I couldn't figure out how to save a worksheet or
plots or other data files and access them in the host OS.

--
Benjamin Jones

[Emil Widmann]

Oh well good news,
this clarification and redirection of "sage on windows" was
overdue ...

I agree with Volker Braun, that a Virtual Box based solution should
have priority.
A viable solution - especially since Virtual Box is GPLed now - is in
short term reach.

My last shot on it was the combined VirtualBox / Sage installer
http://boxen.math.washington.edu/home/emil/doc/html/en/Windows_Installer.html
this image has still some shortcommings compared to Mr. Steins
specifications.
Most notably mouse integration (i.e. VB Guest additions) and autostart
of the browser (minor issue).

a proper usage of the Virtual Box commandline tools would even give
some additional possibilites (e.g. running in headless mode of the VM
and autostart the windows browser).

All in all I think if there would be a dedicated sage days, with some
windows users/sysadmins/programmers
and some sage folks it would be enough to produce a user friendly VM
which meets or even exceeds Mr. Steins specifications.

The Limit of 600 MB for the image will be hard to reach with a full
sized Sage. Sage has grown quite a bit in size over the last 2 years.
But it is possible.

coLinux is nice, but more complicated to install - You also need the X
server.
In my tests coLinux was also slower than the Virtual Machine - read
about it here (There is also a precompiled Sage disk image for
"andLinux")
http://boxen.math.washington.edu/home/emil/doc/SageWin/Sage_on_andLinux.html

I am also still proud of the Sage Live CD, but it is no substitute for
a more integrated solution - The dual boot exe installer will never
win over a majority of users, it's more a geek thing.

Good Luck!
emil

[P Purkayastha]

On Wednesday, January 11, 2012 11:26:43 AM UTC+8, Benjamin Jones wrote:

After a minute or two I couldn't figure out how to save a worksheet or
plots or other data files and access them in the host OS.

--
Benjamin Jones

For this it would be best if shared folders are used, as Keshav mentioned earlier. One shared folder for .sagenb and another shared folder for "My Documents\sagenb" so that the user can right click on images and save it in the shared folder.

[mmarco]

I have tried the new VM from Volker (on a linux host), and the guest
browser works fine, but i can't connect from a host browser. Somebody
else noticed this?

As for the shared folders, i think it would be a good idea to include
a dialog in Emil's installer to configure them after the .ova image is
imported.

[Dima Pasechnik]

I tried running this on a Windows 7 (64-bits), using Virtual Box 4.1.8.

I imported the ova file with the default settings, but the thing fails to start, 

saying 

"Failed to open a session for VM...,

 VT-x features locked on unavailable in MSR

(VERR_VMX_MSR_LOCKED_OR_DISABLED)"

What am I doing wrong?

[Volker Braun]

The virtual machine has hardware virtualization support enabled. You can probably switch on HW virtualization in the BIOS of your computer, which is recommended if you do any virtualization. You can also disable it in the settings of the virtual machine (Settings->System->Acceleration) at a slight performance cost.

[Emil Widmann]

I think the default settings for a 32-bit VM should support the
broadest possible range of
machines - so I would suggest a sage compiled with "FAT_BINARIES" and
HW Acceleration off.

There could be of course a possible sage windows control center gui,
where it is possible to enable HW accelaration (among possible other
customizations and with link to documentation).

[Dima Pasechnik]

Settings->System->Acceleration way worked (I also needed to set the # of CPUs to 1). (For BIOS settings I need to go the the office and attach a monitor to the box :-))

Somehow, I expected I could run the notebook server in the VM and connect to it from the host system, but I don't see a way to do it.

[William Stein]

That should be possible, but with VirtualBox you either have to use
port forwarding (as I recently learned in this thread!) or setup an
*additional* "host only" network interface.

With VMware (and Parallels) the default standard network connection
somehow does both NAT and "host only" networking at the same time,
which is much, much less confusing and user friendly. I guess the
VirtualBox developers never figured out how to do this, or (more
likely) have philosophical objections. Anyway, that could be the
source of your confusion.

-- William

[mmarco]

The port forwarding seems to be well configured in Volker's VM (the
same configuration works fine in otyher VM's), but i don't know why it
does not work. Maybe something isn't right in the guest system?

Volker, are you sure that your guets system is listening to ssh
connections, and that it accepts http connections from the host
system?

[Volker Braun]

On Wed, Jan 11, 2012 at 10:22 AM, Dima Pasechnik <dim...@gmail.com> wrote:

> Somehow, I expected I could run the notebook server in the VM and connect to

> it from the host system, but I don't see a way to do it.

You should be able to connect from the host to the guest port 2222 t(ssh into the virtual machine) and port 8000 (the notebook browser). If you can't then there is something wrong with networking either in the virtual machine or with the host firewall.

On Wed, Jan 11, 2012 at 1:27 PM, William Stein <wst...@gmail.com> wrote:

> With VMware (and Parallels) the default standard network connection

> somehow does both NAT and "host only" networking at the same time,

> which is much, much less confusing and user friendly.

I don't understand what you mean. In VirtualBox, the difference between NAT and Host only is that the packets do not / do go through the host loopback interface. So with Host only different VMs can communicate and you can sniff the traffic, and thats about it.

[William Stein]

To make sure we are on the same page:  have you used VMware much?

[Volker Braun]

On Wednesday, January 11, 2012 3:52:37 PM UTC-5, William wrote:

To make sure we are on the same page:  have you used VMware much?

I haven't. Whenever I wanted to give it a try I eventually found out that my kernel is too new for VMware.

[mmarco]

Volker, can you access the sage notebook from the host using your last
VM? I can't.

[mmarco]

In fact, i get that only the lo interface is up in the guest system. I
think that explains the problem.

[Volker Braun]

It works for me. Since it doesn't work for everybody I'm assuming that VirtualBox presents different bios names for the nic, perhaps depending on the version. I'll change the VM to fix that.

[Dima Pasechnik]

On Thursday, 12 January 2012 04:48:39 UTC+8, Volker Braun wrote:

On Wed, Jan 11, 2012 at 10:22 AM, Dima Pasechnik <di...@gmail.com> wrote:

> Somehow, I expected I could run the notebook server in the VM and connect to

> it from the host system, but I don't see a way to do it.

You should be able to connect from the host to the guest port 2222 t(ssh into the virtual machine) and port 8000 (the notebook browser).

OK, this works. What are the login/password for ssh?

Also, it's not clear why after minimizing the browser window which pops up in the VM, I don't see anything there, just the blank black window.

Does this indicate that something is wrong with the X11 setup on the VM?

If you can't then there is something wrong with networking either in the virtual machine or with the host firewall.

[Volker Braun]

password is always "sage"

I've configured the window manager to the bare minimum. Ideally we'd run without a wm but that seems to break VirtualBox seamless mode. We should also disable moving and minimizing the browser window inside the VM.

[mmarco]

Maybe just running the browser in full screen mode would be enough.
Although, chrome has the problem that it does not show the tabs in
fullscreen mode.

[Keshav Kini]

Maybe you can use a tiling WM, like for example xmonad. It is by
default impossible to minimize any windows, and if I'm not mistaken
it's configurable to not allow de-tiling (i.e. resizing, moving) as
well. This could get hairy if any other windows would ever need to pop
up in the VM, though. At least I can attest to Virtualbox seamless
mode working fine with xmonad.

-Keshav

----
Join us in #sagemath on irc.freenode.net !

[Dima Pasechnik]

On Friday, 13 January 2012 06:40:18 UTC+8, Keshav Kini wrote:

On Fri, Jan 13, 2012 at 06:27, mmarco <mma...@unizar.es> wrote:
> On 12 ene, 16:38, Volker Braun <vbrau...@gmail.com> wrote:
>> I've configured the window manager to the bare minimum. Ideally we'd run
>> without a wm but that seems to break VirtualBox seamless mode.

Do you mean that one cannot run Virtual Box without X11?

I'd imagine if it were possible one can just run the sagenb server there, and

use the browser on the host system only.

[Emil Widmann]

> Do you mean that one cannot run Virtual Box without X11?
> I'd imagine if it were possible one can just run the sagenb server there,
> and
> use the browser on the host system only.

The discussion goes in circles ;-)
Running the Vm without X and accessing it from the host-browser is the
current solution.

If networking/port forwarding from VM to host would work reliably or
even if one would have good testers base and docs about what can go
wrong and how to fix it
then headless mode and autostart of the notebook in the host-browser
would be a good solution.

Problems start when something goes wrong. In headless mode there is
even no visible Linux terminal window.
If if a terminal is brought up then the user is left to the linux
commandline and - probably - get lost.
Example: http://groups.google.com/group/sage-devel/browse_thread/thread/a4808f2cf5b8b79f/03db2e576b21f42e
Hmm, reading this again the missing terminal in headless mode is
probably an asset.

In my personal opinion the work follows 3 tangents:
1) Docs!
2) Integration and support from the windows side , e.g. start-
scripts&gui, menu entries, generation of desktop icons, diagnose
scripts - e.g. network connection/firewall, even eventually combined
installer "VirtualBoxOSE plus Sage VM"
3) Two virtual images, with possibly complete capabilities (e.g. X
+browser, GuestAdditions, graphics in R, JRE, Matplotlib graphic
backend, etc.) which can be started either in server mode (headless)
with autostart of the host browser, or in VirtualBox seamless mode.
One of the sage images is basic, 32/bit, compiled with option FAT
Binaries and 1 processor. The other is 64 Bit and more demanding on
the Hardware.

I tried Volkers VM, but unfortunately it is to heavy to run on my
machine properly (yes, only 1 GB RAM). I also tried to improve my own
Puppy-VM with the Guest Additions, but I failed to get the mouse-
pointer integration to work - tough luck. Personally I will not have
time to work on this a lot over the next several months, and my
opinion is, that the progress is determined if a group of people on
the windows side is motivated to help, be it testing or even
developing. This could be the hardest part, because most windows-users
expect something working out of the box and give up rather quick if
something behaves unexpected. The very few "bug reports" I got mostly
were as detailed as " A is not working and B is not working and I
wasted 15 min on this crappy software". Still better than no feedback
at all ...

Since it the whole is technically nontrivial but definitely not rocket
science I wonder if this would be a possible student class project of
some Computer Science / Engineering course?

just my 2 Cents
emil

[Dima Pasechnik]

On Friday, 13 January 2012 18:22:33 UTC+8, Emil Widmann wrote:

> Do you mean that one cannot run Virtual Box without X11?
> I'd imagine if it were possible one can just run the sagenb server there,
> and
> use the browser on the host system only.

The discussion goes in circles ;-)
Running the Vm without X and accessing it from the host-browser is the
current solution.

Really?! I thought I saw the browser running inside the VM window...

Can't check this right now, perhaps it was just a coincidence that the host system browser 

had neatly placed itself over the VM window... 

[Volker Braun]

Emil was referring to the VM that we currently distribute by "current solution", not the experimental one that I made for this thread. The new one has the browser running inside the VM window. I think I fixed the networking issue, I'll make another alpha version next week or so.

[mmarco]

Working in my own VM, i have noticed a problem of the "browser inside
the VM" aproach: keyboard layout. Is there a way to make VirtualBox
use the host layout inside the guest?

[Emil Widmann]

With my current VM (Live CD) I had a flag that brings up a menu to
choose locals at first run, but that's not an OOB solution.
With the VBoxManage command there is the the possibility to send data
from the host to the guest, though I never used that before. So maybe
it is possible to send the windows keymap settings and use that in the
VM.

If networking is stable I would prefer headless mode with browser from
windows. The browser in the VM would be a fallback if something goes
wrong.

[Dima Pasechnik]

perhaps instead of opening a browser in VM, we should rather open a terminal window there, maybe with Sage started? Then the user might do notebook() there and try accessing sagenb from the browser on the host system.

For the 1st run, this should be OK, IMHO.

How to customize the subsequent runs, can be discussed.

[Emil Widmann]

> perhaps instead of opening a browser in VM, we should rather open a
> terminal window there, maybe with Sage started?

This is exactly the current solution (The version which we currently
distribute).
The whole discussion is about improving the Out of the Box experience,
where especially the 1st run is very important.

[mmarco]

I have been working on a VM with compressed filesystems. I have
thought that maybe a good aproach to the "automatic generation"
problem would be to mount the sage directory from a compressed file in
a shared folder. It should need, as William sugested, a windows GUI
app that would take care of configuring shared folders properly. But
that way, producing a new version of the VM after a new release of
Sage would be as easy as getting an installation of sage (which works
fine in the guest os) and compressing it to a file.

In my tests, the Sage folder takes around 2 gigs of disk space (after
make micro_release, which by the way gave me an error). Compressing it
into a squashfs file with lzma reduces it to 760 megs. And we have to
add the rest of the system (almost half a gig in my system, also
squashed). So, even with compressed filesystems, the apliance would
take a lot of disk space.

[Emil Widmann]

On Feb 4, 8:25 pm, mmarco <mma...@unizar.es> wrote:
> I have been working on a VM with compressed filesystems. I have
> thought that maybe a good aproach to the "automatic generation"
> problem would be to mount the sage directory from a compressed file in
> a shared folder. It should need, as William sugested, a windows GUI
> app that would take care of configuring shared folders properly.
> But
> that way, producing a new version of the VM after a new release of
> Sage would be as easy as getting an installation of sage (which works
> fine in the guest os) and compressing it to a file.

This would also an alternative possibility for a version upgrade - the
new version could just be downloaded replace the old file in the vm.

> In my tests, the Sage folder takes around 2 gigs of disk space (after
> make micro_release, which by the way gave me an error). Compressing it
> into a squashfs file with lzma reduces it to 760 megs. And we have to
> add the rest of the system (almost half a gig in my system, also

> squashed). So, even with compressed filesystems, the appliance would

> take a lot of disk space.

I think the consensus here is that disk space is a minor trade off. I
guess with the right toolchain you could get the appliance with full
sage, compilers, maybe even virtualbox bundled down to around 800 MB
(see as an example the last porteus 64 bit live cd of sage). So your
build with ~ 1.3 GB is not to far off.

What about documentation? I would make the windows documentation an
extra package which can be downloaded as needed (and viewed from
windows, or either just refer to the web documentation).

A more technical detail - I am curious:
How to you allow persistant changes to the sage directory in the
compressed filesystem? Do you plan to use an overlayed filesystem with
a persistant savefile? Or do you have the sage directories just as
read only?
cheers

[Volker Braun]

I think thats all premature optimization. First we need a bulletproof way of running Sage in the VM. At the end we can still worry about ways to shave off another megabyte or two. Most VM questions I got were about how to add more stuff to the VM within its space constraints. Nobody had an issue with it requiring a couple of gigabytes of disk space.

[mmarco]

> A more technical detail - I am curious:
> How to you allow persistant changes to the sage directory in the
> compressed filesystem? Do you plan to use an overlayed filesystem with
> a persistant savefile? Or do you have the sage directories just as
> read only?
> cheers

I use unionfs to mix the squashed filesystem with a directory to save
the changes. Maybe that would be a source of errors if somebody makes
changes in his sage directory and then upgrade by replacing the
compressed file. That should be tested, but i think that the kind of
user that makes changes to his sage install wouldn't be too lost if he
is asked to peform some operations inside the VM before a direct
upgrade.

[mmarco]

I agree that this is not the most important thing right now. But
incidentally, working on this gave me the idea of separating the guest
OS from the sage install in different files, which, i think, is an
approach to the "automatic generation" that is worth considering.

But yes, you are right. The most important thing now is to find
someone that would write the windows gui to take care of the VM, and
then a lot of testers to detect the possible ways it could fail.

[Emil Widmann]

On Feb 5, 8:30 am, Volker Braun <vbraun.n...@gmail.com> wrote:
> I think thats all premature optimization. First we need a bulletproof way
> of running Sage in the VM.

I agree - "premature optimization, the root of all evil". In fact the
VM itself is just one module of the interface and it is definitely far
better developed than other parts. However seeing how many people here
build VM's at all I thought it would be nice to give some interested
feedback to mmarco.

About those other parts: Really, the Interface from the windows side
has a lot of potential. All this stuff typical windows users expect
when installing new software like desktop icons, menu entries,
installer, on screen help / documentation Pop-ups... boring stuff like
that. The base could and should be the current VM for now (or
eventually your last beta with X).

I tried to adress some of those issues with the combined Sage/
Virtualbox Windows installer Amongst those were: Autoinstallation and
autoimport of the VM to Virtualbox, autogeneration of startscripts and
entries in the windows Program Folder, Communication with Vbox over
the VBoxManage command. Maybe with more testing it could be even
possible to stabilize the network connection from windows and VM to a
point where it is acceptable and the overhead of running X in the VM
wouldn't be necessary. My personal preference would be to hide the
whole Virtualbox interface from the normal user completely (headless
mode) but have an expert, debug, or development mode available which
could bring up the Virtual Box Window.

> shave off another megabyte or two"

That remark is a bit terse and misleading. I had Sage VM's built below
400 MB with most of the necessary specs fulfilled (yes, even Java for
3d- and R-plotting). But that can only be achieved if the architecture
& design of the base system is adapted from the beginning - seeing it
just as "shaving off" will lead you to the conclusion that it is not
worth the effort because then you just can achieve the 1 or 2 MB you
mentioned.

Personally I will not have time to do more "work" on this until -
maybe- October, but I will "lurk" around a bit and follow whats going
on.
cheers
emil

[Dima Pasechnik]

there is no need for "custom" gui: plenty of tools around allows one to build installers doing nontrivial things.

One such thing 15 years ago was InstallShield, they seem to be still in business.

A free one is Innosetup: http://www.jrsoftware.org/isinfo.php

Cygwin has its own GUI installer written from scratch.

M$ has its own Windows Installer (which was, and probably still is, a nightmare, as usual with M$ products).

Dima

Do not forget  that you ideally also want updates to be done more or less automatically.

  

[Emil Widmann]

I used not installshield but the opensource "NSIS installer" - I
didn't program a custom gui. Still the interface to Virtualbox has to
be "programmed" but it is fairly easy, mostly 1 or 2 liner scripts.
The upgrade to a new sage version should be as easy as to replace the
VM file.

[Volker Braun]

On Sunday, February 5, 2012 2:59:40 AM UTC-8, Emil Widmann wrote:

About those other parts: Really, the Interface from the windows side
has a lot of potential. All this stuff typical windows users expect
when installing new software like desktop icons, menu entries,
installer, on screen help / documentation Pop-ups... boring stuff like
that. The base could and should be the current VM for now (or
eventually your last beta with X). 

I was recently thinking about our options for a Sage launcher program This would be a relatively small program that controls VirtualBox, hiding the normal VirtualBox gui. This is pretty easy using the VBoxManage utility. The Sage launcher program could then

  * download new Virtual Machines if necessary

  * let the user choose between different versions of the Sage VM

  * diagnose port / firewall issues

  * automatically export notebooks from the VM using a bundled ssh client

  * write the launcher in Python (which we also include) using some Python GUI toolkit (included, too)

[William Stein]

Big +1.

I'm curious if anybody else has ever written such a program for
another project? If anybody out there is particularly good at
searching for such things, could you look? It's not inconceivable
there's something out there we can learn from.

-- william

>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[Emil Widmann]

@mmarco

>I use unionfs to mix the squashed filesystem with a directory to save
>the changes. Maybe that would be a source of errors if somebody makes
>changes in his sage directory and then upgrade by replacing the
>compressed file.

I think the unionfs is a well tested technology now, but a clean
solution would be that in case of such an replacement/upgrade of the
sage squashfs the whole sage directory tree in the save directory is
deleted too - then the user starts with a "fresh" install.

> > I was recently thinking about our options for a Sage launcher program This
> > would be a relatively small program that controls VirtualBox, hiding the
> > normal VirtualBox gui. This is pretty easy using the VBoxManage utility. The
> > Sage launcher program could then
> >   * download new Virtual Machines if necessary
> >   * let the user choose between different versions of the Sage VM
> >   * diagnose port / firewall issues
> >   * automatically export notebooks from the VM using a bundled ssh client
> >   * write the launcher in Python (which we also include) using some Python
> > GUI toolkit (included, too)

Sounds great!
This could be just a small application in the beginning with the sage
logo and a start button , but can be expanded later.
I am not sure if it is necessary to write this in python - that is
another dependency.
Why not use VBscript or precompile it? Distribution of binaries on
windows should be no problem.

> Big +1.
>
> I'm curious if anybody else has ever written such a program for
> another project?   If anybody out there is particularly good at
> searching for such things, could you look?  It's not inconceivable
> there's something out there we can learn from.
>
>  -- william

Essentially the documentation of the vboxmanage command shows, that a
very detailed control of the VM is possible.
As an example the commands "VBoxManage controlvm <VM> setlinkstate/
nic" lets you switch on/off network or switch to different network
types like "nat/bridged"
Or the command "VBoxManage guestcontrol execute" lets you start
programs on the guest from the hostmachine.

As I am on it let me share some thoughts - some of those may be in the
category "premature optimization", but just ignore those you don't
like:

Autoimport of virtual machines "ovas" is possible, however it might be
worth considering to not "import" machines, but to create the VM at
runtime and attach the VM on premade bootable vdi harddisk.
Advantages: it is possible to create the machine according to the host
specs (memory, processor). Attaching disks is much faster than
importing ovas (can take minutes).
Disadvantage: Autoimport ova is tested - autocreation of VM and
attaching of virtual disk is not coded and not tested (maybe 20 lines
of VBScript code)

Set the Virtual machine Folder to a path where all users can reach the
sage VM . Currently the Virtualbox default is to store the ova in the
user space, so other accounts cannot use the sage VM. We had at least
one support/ask sage question about this. A possible solution would be
to have the Virtual Machine available public, but use VBoxManage /
Guestadditions to mirror an unique .sage folder in each user
directory.

The hardware specs of the host machine (processor) are mapped to the
VirtualBox - so what policy should apply here? - There could be 1
minimal machine which was compiled for low specs (32 bit,
FAT_BINARIES, 1 processor ...), so that it should run on widest range
of possible hardware. And maybe a 64bit version with complete modern
processor instruction set and more turbo specs available.

I found the following links:
http://www.virtualbox.org/manual/ch08.html
Example GUI: https://github.com/c0state/VirtualBox-Snapshot-Deletion-GUI
web interface: http://code.google.com/p/phpvirtualbox/
commercial: http://onapp.com/cloud/how-it-works/control-panel/
http://en.wikipedia.org/wiki/VBScript
Python Compiler(?): http://nuitka.net/blog/
VirtualBox/Multiuser environment: http://vu1tur.eu.org/vboxctrl

cheers

[Volker Braun]

On Monday, February 6, 2012 1:20:13 AM UTC-8, Emil Widmann wrote:

Why not use VBscript or precompile it? Distribution of binaries on
windows should be no problem.

A launcher written in Python could be used on other OS'es, too. And VBscript is an ugly abomination, nobody in his right mind will voluntarily use it. Not to mention that we have lots of people with Python skills.

[Keshav Kini]

On Mon, Feb 6, 2012 at 17:20, Emil Widmann <emil.w...@gmail.com> wrote:
> I think the unionfs is a well tested technology now, but a clean
> solution would be that in case of such an replacement/upgrade of the
> sage squashfs the whole sage directory tree in the save directory is
> deleted too - then the user starts with a "fresh" install.

That's what I think too, but it disagrees with the current behavior of
`sage -upgrade`, which tries to merge any committed changes into the
new version you're upgrading to. That doesn't make sense IMO but there
you are.

> Sounds great!
> This could be just a small application in the beginning with the sage
> logo and a start button , but can be expanded later.
> I am not sure if it is necessary to write this in python - that is
> another dependency.
> Why not use VBscript or precompile it? Distribution of binaries on
> windows should be no problem.

Python programs can be made into Windows binaries with py2exe_, a
distutils extension. GUIs can be made relatively easily with PySide_,
or so I've heard - haven't gotten around to trying it myself yet.

.. _py2exe: http://py2exe.org/
.. _PySide: http://pyside.org/

I wonder why you suggest using VBscript. It's not exactly a majority /
serious language among Windows developers as I recall and certainly
nobody on Mac or Linux uses it. I think we should stick to Python if
possible for widest familiarity among our developers. Maybe you just
meant that it is a built-in scripting language in Windows, which is
true, but as you said, there's no problem in shipping binaries for
Windows.

[Emil Widmann]

> A launcher written in Python could be used on other OS'es, too.

good point

> VBscript is an ugly abomination, nobody in his right mind will voluntarily
> use it.

Lots of people (windows folks) use it even voluntarily - I will not
speculate on their (or my) state of mind ...

>Not to mention that we have lots of people with Python skills.

good point

My first thought was it might be overkill to ship a complete
programming language and a Gui frontend for the sake of writing a
small launcher application (or "rewrite" parts of the VirtualBox
control GUI). But OK, this goes into the same line as the "size of the
VM" discussion - I won't argue about pulling in additional 1 or 2
MB's ;-).

For anybody interested, this is the code (VBscript ) of the NSIS Sage-
Virtualbox Windows installer.
http://boxen.math.washington.edu/home/emil/Win-Inst/SageWinInstBuildDir/main.nsi
It mostly brings up some Messageboxes or calls VBoxManage with
"ExecWait", no rocket science involved. So it might be an abomination
but one could manage.

[mmarco]

I have little experience with pyqt, and i am not sure that would be
the way to go. A windows user that would want to use a pyqt program
would need to have installed in his system: python, pyqt and qt. The
offline windows installer of qt is 1.3 gigs. That's overkill for a
simple gui app. There must be a simpler way to do it.

I haven't used pyside, but for what i see in the documentation, it
follows the same approach than pyqt.


On 6 feb, 12:22, Keshav Kini <keshav.k...@gmail.com> wrote:

[Keshav Kini]

On Mon, Feb 6, 2012 at 20:03, mmarco <mma...@unizar.es> wrote:
> I have little experience with pyqt, and i am not sure that would be
> the way to go. A windows user that would want to use a pyqt program
> would need to have installed in his system: python, pyqt and qt. The
> offline windows installer of qt is 1.3 gigs. That's overkill for a
> simple gui app. There must be a simpler way to do it.
>
> I haven't used pyside, but for what i see in the documentation, it
> follows the same approach than pyqt.

This is 100% avoidable. Here is an example of a standalone Windows
executable which is written in Python using PyQt: http://ankisrs.net/
After installation it is a couple dozen megabytes, and this is a
full-featured application that does a bunch of stuff including
audio/video playback, HTML rendering, network connectivity, sqlite
data storage, etc. etc.

I'm not sure exactly how it is done but I assume it uses some
combination of py2exe and Qt tools. (Taking a look in the source code
should prove enlightening: http://github.com/dae/ankiqt ). After
installation there are some Qt-named DLL files sitting in the
program's directory. You absolutely do not need the Qt SDK, or a copy
of Python, or a copy of PyQt to run such an application. To develop
the application is a different story of course.