Lorien Labs: an IDE for Sage

[Alejandro Serrano Mena]

Hello,
For the past few months I've been working on a prototype of an Integrated Environment for developing and running Sage. I decided to start the project for mainly these reasons:
- There is no environment for developing Sage programs or libraries: nowadays the Notebook is not the best option for those taks
- Allowing Sage to run in local environments: that means that we can have more information that the Notebook provides: better completion, using functions between different files...

As of now you can download the code with "svn co https://lorienlabs.svn.sourceforge.net/svnroot/lorienlabs/trunk lorienlabs". The only problem is that PyQt4 is neccessary, but must be compiled inside the Sage environment. I've uploaded instructions on how to do this in "doc/readme" in the SVN repository.

Lorien Labs (the name has to be changed, I know) now only emulates notebook worksheets, with an added docklet to show the variables in a sheet and their contents. It also includes a preview of how a new viewer can be added to the program with a graph viewer (type graphs.DodecaedralGraph() and press Shift + Enter to see what I'm talking about). I'm currently planning how to add support for Sage code files, with autocompletion and so on, and how a worksheet may access the functions on those code files.

Any suggestion and help is welcome :) I hope you like it.

Alejandro

[Carlo Hamalainen]

On Mon, Nov 16, 2009 at 11:25 AM, Alejandro Serrano Mena
<tru...@gmail.com> wrote:
> Any suggestion and help is welcome :) I hope you like it.

I'm trying to compile with Ubuntu 9.04 and I get to QScintilla and
have a problem:

carlo@eeepc:~/sage/qt/QScintilla-gpl-2.4/Python$ ./../../../sage
-python configure.py
Error: Qsci/qsciglobal.h could not be found in /usr/include/qt4. If QScintilla
is installed then use the -n argument to explicitly specify the correct
directory.

I Googled and couldn't find a fix. Have you seen this error?

Thanks,

--
Carlo Hamalainen
http://carlo-hamalainen.net

[Alejandro Serrano Mena]

It usually means that you need to install libqscintilla2-dev and libqt4-dev, so development files are used.

2009/11/16 Carlo Hamalainen <carlo.ha...@gmail.com>

[William Stein]

On Mon, Nov 16, 2009 at 5:40 AM, Alejandro Serrano Mena
<tru...@gmail.com> wrote:
> It usually means that you need to install libqscintilla2-dev and libqt4-dev,
> so development files are used.

1. What Linux distribution do you use to do development on this (I
have access to most of them, so I might as well test with the same as
you)?

2. Can you post some screenshots (or screencasts)?

3. "Lorien Labs now only emulates notebook worksheets," What does that
mean? Does that mean I can load an existing Sage worksheet (a sws
file), use it in Lorien Labs, then save it to sws and use that in the
Sage notebook? If so -- bravo -- I think that would be *excellent*.

4. "Lorien Labs" (the name): As mentioned above, why don't we aim for
compatibility between the web-based Sage notebook sws files and the
Lorien Labs-based notebooks. Moreover, we could make the name
compatible as well. E.g., Could we call it something like "Sage
Notebook QT" or the "QT Sage Notebook" (pronounced "cute Sage
notebook") or "SageQT" ?

>
> 2009/11/16 Carlo Hamalainen <carlo.ha...@gmail.com>
>>
>> On Mon, Nov 16, 2009 at 11:25 AM, Alejandro Serrano Mena
>> <tru...@gmail.com> wrote:
>> > Any suggestion and help is welcome :) I hope you like it.
>>
>> I'm trying to compile with Ubuntu 9.04 and I get to QScintilla and
>> have a problem:
>>
>> carlo@eeepc:~/sage/qt/QScintilla-gpl-2.4/Python$ ./../../../sage
>> -python configure.py
>> Error: Qsci/qsciglobal.h could not be found in /usr/include/qt4. If
>> QScintilla
>> is installed then use the -n argument to explicitly specify the correct
>> directory.
>>
>> I Googled and couldn't find a fix. Have you seen this error?
>>
>> Thanks,
>>
>> --
>> Carlo Hamalainen
>> http://carlo-hamalainen.net
>>
>>
>
>
> >
>

--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

[Alejandro Serrano Mena]

Hello again,

2009/11/16 William Stein <wst...@gmail.com>

On Mon, Nov 16, 2009 at 5:40 AM, Alejandro Serrano Mena
<tru...@gmail.com> wrote:
> It usually means that you need to install libqscintilla2-dev and libqt4-dev,
> so development files are used.

1. What Linux distribution do you use to do development on this (I
have access to most of them, so I might as well test with the same as
you)?

I was using Ubuntu 9.04 and currently I'm using Ubuntu 9.10, with the latest components of SIP, PyQt and PyQScintilla.
For Qt, a version 4.5 or greater is needed (because the WebKit component used to render the outputs is not available in previous versions). You also need PyQt4 with the same minor version as your Qt installation (that is, if you have Qt 4.5.1 in your system, you need PyQt 4.5.x for it to work).

2. Can you post some screenshots (or screencasts)?

Sure! I'm uploaded them into the Sourceforge site: https://sourceforge.net/project/screenshots.php?group_id=287015
It shows the current features: autocompletion, variable explorer, graph viewer, plots...

3. "Lorien Labs now only emulates notebook worksheets," What does that
mean?  Does that mean I can load an existing Sage worksheet (a sws
file), use it in Lorien Labs, then save it to sws and use that in the
Sage notebook?   If so -- bravo -- I think that would be *excellent*.

As of now you cannot save a worksheet. The idea is double:
- First of all, use the same sws format as the notebook, so files would be completely interoperable
- Be able to connect this Qt frontend to a running instance of Sage Notebook and directly use the files from there. In that way we could get automatic revisions, sharing... that Sage Notebook has.

4. "Lorien Labs" (the name):  As mentioned above, why don't we aim for
compatibility between the web-based Sage notebook sws files and the
Lorien Labs-based notebooks.   Moreover, we could make the name
compatible as well.  E.g., Could we call it something like "Sage
Notebook QT" or the "QT Sage Notebook" (pronounced "cute Sage
notebook") or "SageQT" ?

Yes, the compatibility would be a great idea!
As for the name, I've been suggested by a friend to use "+Sage", which is Spanish is read "masaje" which means massage. But any other name is great for me :)

For the future, what do you think would be the best way to allow people to write entire Sage code files inside the environment and then test them with the QT Notebook? Has Sage got any facility to load Sage files dinamically (I don't know whether using a plain import works)?

[William Stein]

On Mon, Nov 16, 2009 at 1:57 PM, Alejandro Serrano Mena

<tru...@gmail.com> wrote:
> Hello again,
>
> 2009/11/16 William Stein <wst...@gmail.com>
>>
>> On Mon, Nov 16, 2009 at 5:40 AM, Alejandro Serrano Mena
>> <tru...@gmail.com> wrote:
>> > It usually means that you need to install libqscintilla2-dev and
>> > libqt4-dev,
>> > so development files are used.
>>
>> 1. What Linux distribution do you use to do development on this (I
>> have access to most of them, so I might as well test with the same as
>> you)?
>
> I was using Ubuntu 9.04 and currently I'm using Ubuntu 9.10, with the latest
> components of SIP, PyQt and PyQScintilla.
> For Qt, a version 4.5 or greater is needed (because the WebKit component
> used to render the outputs is not available in previous versions). You also
> need PyQt4 with the same minor version as your Qt installation (that is, if
> you have Qt 4.5.1 in your system, you need PyQt 4.5.x for it to work).

Are these -- PyQt4, QT 4.5.1, etc. -- all standard Ubuntu packages?
Or did you have to build them from source?

>> 2. Can you post some screenshots (or screencasts)?
>
> Sure! I'm uploaded them into the Sourceforge site:
> https://sourceforge.net/project/screenshots.php?group_id=287015
> It shows the current features: autocompletion, variable explorer, graph
> viewer, plots...

Nice.

>>
>> 3. "Lorien Labs now only emulates notebook worksheets," What does that
>> mean?  Does that mean I can load an existing Sage worksheet (a sws
>> file), use it in Lorien Labs, then save it to sws and use that in the
>> Sage notebook?   If so -- bravo -- I think that would be *excellent*.
>
> As of now you cannot save a worksheet. The idea is double:
> - First of all, use the same sws format as the notebook, so files would be
> completely interoperable
> - Be able to connect this Qt frontend to a running instance of Sage Notebook
> and directly use the files from there. In that way we could get automatic
> revisions, sharing... that Sage Notebook has.

I see.

How does this Qt frontend communicate with the Sage process? Are they
both the same process and the Sage compute thread is just another
thread? Or do you use some other form of communication?

>>
>> 4. "Lorien Labs" (the name):  As mentioned above, why don't we aim for
>> compatibility between the web-based Sage notebook sws files and the
>> Lorien Labs-based notebooks.   Moreover, we could make the name
>> compatible as well.  E.g., Could we call it something like "Sage
>> Notebook QT" or the "QT Sage Notebook" (pronounced "cute Sage
>> notebook") or "SageQT" ?
>
> Yes, the compatibility would be a great idea!
> As for the name, I've been suggested by a friend to use "+Sage", which is
> Spanish is read "masaje" which means massage. But any other name is great
> for me :)
>
> For the future, what do you think would be the best way to allow people to
> write entire Sage code files inside the environment and then test them with
> the QT Notebook? Has Sage got any facility to load Sage files dinamically (I
> don't know whether using a plain import works)?

If a file is named foo.sage just do

sage: load foo.sage

There is also an attach feature. These just preparse the file then
use the standard Python execfile (or exec) feature.

[Alejandro Serrano Mena]

2009/11/17 William Stein <wst...@gmail.com>

On Mon, Nov 16, 2009 at 1:57 PM, Alejandro Serrano Mena

<tru...@gmail.com> wrote:
> Hello again,
>
> 2009/11/16 William Stein <wst...@gmail.com>
>>
>> On Mon, Nov 16, 2009 at 5:40 AM, Alejandro Serrano Mena
>> <tru...@gmail.com> wrote:
>> > It usually means that you need to install libqscintilla2-dev and
>> > libqt4-dev,
>> > so development files are used.
>>
>> 1. What Linux distribution do you use to do development on this (I
>> have access to most of them, so I might as well test with the same as
>> you)?
>
> I was using Ubuntu 9.04 and currently I'm using Ubuntu 9.10, with the latest
> components of SIP, PyQt and PyQScintilla.
> For Qt, a version 4.5 or greater is needed (because the WebKit component
> used to render the outputs is not available in previous versions). You also
> need PyQt4 with the same minor version as your Qt installation (that is, if
> you have Qt 4.5.1 in your system, you need PyQt 4.5.x for it to work).

Are these -- PyQt4, QT 4.5.1, etc. -- all standard Ubuntu packages?
Or did you have to build them from source?

QT and PyQT are standard Ubuntu packages. You need both installed to run the QT environment. However you also need a version of PyQT (and only PyQT, not QT) compiled under the Sage environment (so it would be installed inside the bin, lib and those folders inside the Sage folders)

>> 2. Can you post some screenshots (or screencasts)?
>
> Sure! I'm uploaded them into the Sourceforge site:
> https://sourceforge.net/project/screenshots.php?group_id=287015
> It shows the current features: autocompletion, variable explorer, graph
> viewer, plots...

Nice.

Thank you.

>>
>> 3. "Lorien Labs now only emulates notebook worksheets," What does that
>> mean?  Does that mean I can load an existing Sage worksheet (a sws
>> file), use it in Lorien Labs, then save it to sws and use that in the
>> Sage notebook?   If so -- bravo -- I think that would be *excellent*.
>
> As of now you cannot save a worksheet. The idea is double:
> - First of all, use the same sws format as the notebook, so files would be
> completely interoperable
> - Be able to connect this Qt frontend to a running instance of Sage Notebook
> and directly use the files from there. In that way we could get automatic
> revisions, sharing... that Sage Notebook has.

I see.

How does this Qt frontend communicate with the Sage process?  Are they
both the same process and the Sage compute thread is just another
thread?   Or do you use some other form of communication?

I use the facilities available in Python. Each worksheet has an associated thread where all the work happens. Each worksheet also saves the state of the variables. When the user wants to execute any code, it is precompiled from Sage to Python code, then the Abstract Syntax Tree is manipulated so the result of the last line of code could be recovered, and then that code is basically executed via exec.

>>
>> 4. "Lorien Labs" (the name):  As mentioned above, why don't we aim for
>> compatibility between the web-based Sage notebook sws files and the
>> Lorien Labs-based notebooks.   Moreover, we could make the name
>> compatible as well.  E.g., Could we call it something like "Sage
>> Notebook QT" or the "QT Sage Notebook" (pronounced "cute Sage
>> notebook") or "SageQT" ?
>
> Yes, the compatibility would be a great idea!
> As for the name, I've been suggested by a friend to use "+Sage", which is
> Spanish is read "masaje" which means massage. But any other name is great
> for me :)
>
> For the future, what do you think would be the best way to allow people to
> write entire Sage code files inside the environment and then test them with
> the QT Notebook? Has Sage got any facility to load Sage files dinamically (I
> don't know whether using a plain import works)?

If a file is named foo.sage just do

  sage: load foo.sage

There is also an attach feature.  These just preparse the file then
use the standard Python execfile (or exec) feature.

Great! Just a few more questions:
- Is there any way to tell Sage which is the "root" directory for searching files. That is, when an user executes "load foo.sage", how can the environment know where to find the file?
- What happens if you load a file twice?

Thank you

[William Stein]

On Tue, Nov 17, 2009 at 12:37 PM, Alejandro Serrano Mena
<tru...@gmail.com> wrote:

>> Are these -- PyQt4, QT 4.5.1, etc. -- all standard Ubuntu packages?
>> Or did you have to build them from source?
>
> QT and PyQT are standard Ubuntu packages. You need both installed to run the
> QT environment. However you also need a version of PyQT (and only PyQT, not
> QT) compiled under the Sage environment (so it would be installed inside the
> bin, lib and those folders inside the Sage folders)
>>

OK.

>> How does this Qt frontend communicate with the Sage process?  Are they
>> both the same process and the Sage compute thread is just another
>> thread?   Or do you use some other form of communication?
>
> I use the facilities available in Python. Each worksheet has an associated
> thread where all the work happens. Each worksheet also saves the state of
> the variables. When the user wants to execute any code, it is precompiled
> from Sage to Python code, then the Abstract Syntax Tree is manipulated so
> the result of the last line of code could be recovered, and then that code
> is basically executed via exec.

What happens if code is actually doing a serious computation (forever)?, e.g.,

sage: n = factorial(10^9)

or the code segfaults (which is easy to do with cython code), e.g.,

sage: cython_lambda('', '(<int*>0)[0]')()

?

William

>> There is also an attach feature.  These just preparse the file then
>> use the standard Python execfile (or exec) feature.
>>
> Great! Just a few more questions:
> - Is there any way to tell Sage which is the "root" directory for searching
> files. That is, when an user executes "load foo.sage", how can the
> environment know where to find the file?

Looking at SAGE_ROOT/devel/sage/sage/misc/preparser.py, there is no
load PATH -- it just uses the current directory. And here's what
loading an actual sage file does:

try:
G = open(name_load)
except IOError:
print "File '%s' not found, so skipping load of
%s"%(name_load, name_load)
i += 1
continue
else:
A = A[:i] + G.readlines() + A[i+1:]
continue

I.e., it's just textual included in the block of code that gets evaluated.

William

> - What happens if you load a file twice?

It gets loaded twice.

William