poll for making dot2tex a standard spkg

[Nicolas M. Thiery]

Dear Sage developers,

This is a poll for making dot2tex a standard spkg.

- [ ] Yes, make dot2tex a standard spkg
- [ ] No, keep it as optional

Description of dot2tex
----------------------

dot2tex [1] is a small pure python module (5000 lines), whose purpose
is to give graphs generated by Graphviz a more LaTeX friendly look and
feel. This is accomplished by converting xdot output from Graphviz to
a series of PSTricks or PGF/TikZ commands.

The current dot2tex spkg weights 900K. It could be trimmed down to
300K by removing PNG's that are only used for testing.

Usage in Sage and rationale for making dot2tex a standard spkg
--------------------------------------------------------------

dot2tex+graphviz are used quite intensively to visualize combinatorial
data like crystal graphs and the like (see e.g. [2]). In fact, that's
one of the very first feature beginners in (algebraic) combinatorics
want to play with. So it would be nice to reduce as much as possible
the prerequisites. Currently they are:

(1) Installing graphviz
(2) Installing dot2tex with sage -i dot2tex
(3) Having a latex distribution that includes a recent enough version
of pgf/tikz (2010 or later?)

Including graphviz into Sage has been discussed a couple times;
however there are a priori licence issues (also the graphviz spkg has
been broken for a while, and nobody stood up to fix that). That's not
so bad because it's usually very easy to install graphviz on one's
operating system; in fact it's often already there.

We can hope that (3) will finally fade out by itself.

Making dot2tex a standard spkg would save on step (2); granted it's a
relatively trivial step, but it often became a burden in practice
because our beginners then needed to have a shell, to have network
access, ...

Making dot2tex standard would also simplify a bit the doctest
"optional" logic: we would only need to mark as optional those tests
that actually require graphviz. Again a small step, but in practice
#14594 would have been detected earlier if dot2tex had been standard.

Cheers,
Nicolas

[1] http://www.fauskes.net/code/dot2tex/
[2] http://wiki.sagemath.org/combinat/CoolPictures

--
Nicolas M. Thi�ry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Dima Pasechnik]

Will it become the 1st standard package that needs an optional package, i.e. graphviz,  to function?

Nicolas M. Thi�ry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Jeroen Demeyer]

On 05/22/2013 04:32 PM, Nicolas M. Thiery wrote:
> - [X] No, keep it as optional

A standard package which is only useful in the presence of an optional
package doesn't make sense to me.

And I fail to see why installing both graphviz and dot2tex is more
difficult than just installing dot2tex (either as optional Sage package
or system-wide).

[kcrisman]

Agreed. 

[Nicolas M. Thiery]

On Wed, May 22, 2013 at 09:13:09AM -0700, kcrisman wrote:
> A standard package which is only useful in the presence of an optional
> package doesn't make sense to me.

It simplifies our users's life, and that is useful! Also it simplifies
*my* life: I am tired, e.g. during Sage Days, of having to explain
about Sage packages to our complete beginners for something that is
often one of the very first feature that they want to play with.

There is plenty of code in the Sage library that only works in the
presence of optional packages. But we ship it standard. Or shall we
extract, e.g. the interface to Maple/Magma/... as optional packages? :-)

A partial answer for Dima: for the notebook spkg to be useful, you
need either a local browser or you want to be running it securely,
which requires the optional pyopenssl spkg. So there already exists a
standard spkg that depends on an optional spkg or an optional system
package.

> And I fail to see why installing both graphviz and dot2tex is
> more difficult than just installing dot2tex (either as optional
> Sage package or system-wide).

(I assume you meant graphviz in this last sentence)

An answer is that graphviz is often already installed on the system.
In which case nothing needs to be done.

Cheers,
Nicolas

PS: one technical question: can a pure python spkg be installed
without any development tools (e.g. without having X-code installed on
MacOS X)?

[William Stein]

On Wed, May 22, 2013 at 9:58 AM, Nicolas M. Thiery
<Nicolas...@u-psud.fr> wrote:
> On Wed, May 22, 2013 at 09:13:09AM -0700, kcrisman wrote:
>> A standard package which is only useful in the presence of an optional
>> package doesn't make sense to me.
>
> It simplifies our users's life, and that is useful! Also it simplifies
> *my* life: I am tired, e.g. during Sage Days, of having to explain
> about Sage packages to our complete beginners for something that is
> often one of the very first feature that they want to play with.

I installed the dot2tex package on https://cloud.sagemath.com, but
when I try to actually use it with this example:

g = sage.categories.category.category_graph()
g.set_latex_options(format="dot2tex")
view(g, pdflatex=True, tightpage = True)

I just get this error:

RuntimeError:
dot2tex not available.

graphviz is installed systemwide via "sudo apt-get install graphviz".

Anyway, I'm surprised that I'm thwarted at using dot2tex to do
anything with Sage on an Ubuntu 12.04 LTS system, which is pretty
standard.

>
> There is plenty of code in the Sage library that only works in the
> presence of optional packages. But we ship it standard. Or shall we
> extract, e.g. the interface to Maple/Magma/... as optional packages? :-)

Maple/Magma/etc. are not optional packages.

> A partial answer for Dima: for the notebook spkg to be useful, you
> need either a local browser or you want to be running it securely,
> which requires the optional pyopenssl spkg.

A network service that only binds to localhost doesn't need ssl.
Also, one can easily (one line) make an *ssh tunnel* to run the
notebook securely on a remote machine without installing pyopenssl.

> So there already exists a
> standard spkg that depends on an optional spkg or an optional system
> package.
>
>> And I fail to see why installing both graphviz and dot2tex is
>> more difficult than just installing dot2tex (either as optional
>> Sage package or system-wide).
>
> (I assume you meant graphviz in this last sentence)
>
> An answer is that graphviz is often already installed on the system.
> In which case nothing needs to be done.

I think graphviz has compiled code, whereas dot2tex is python code, so
graphviz could be hard to install.

Also relevant is that graphviz has a nasty GPL-incompatible license,
so we can never ever include it with Sage, whereas dot2tex has a very,
very standard open license:

https://code.google.com/p/dot2tex/source/browse/LICENSE

>
> Cheers,
> Nicolas
>
> PS: one technical question: can a pure python spkg be installed
> without any development tools (e.g. without having X-code installed on
> MacOS X)?
>
> --

> Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
> http://Nicolas.Thiery.name/
>
> --
> You received this message because you are subscribed to the Google Groups "sage-combinat-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-combinat-d...@googlegroups.com.
> To post to this group, send email to sage-comb...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-combinat-devel?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[Nicolas M. Thiery]

On Wed, May 22, 2013 at 10:39:36AM -0700, William Stein wrote:
> I installed the dot2tex package on https://cloud.sagemath.com, but
> when I try to actually use it with this example:
>
> g = sage.categories.category.category_graph()
> g.set_latex_options(format="dot2tex")
> view(g, pdflatex=True, tightpage = True)
>
> I just get this error:
>
> RuntimeError:
> dot2tex not available.
>
> graphviz is installed systemwide via "sudo apt-get install graphviz".

Thanks for the report. Which version of Sage? which version of the
dot2tex spkg?

Sage 5.10 beta requires a new dot2tex package (due to the upgrade of
matplotlib); this is being resolved in #14594 (and actually the source
of this discussion).

> > There is plenty of code in the Sage library that only works in the
> > presence of optional packages. But we ship it standard. Or shall we
> > extract, e.g. the interface to Maple/Magma/... as optional packages? :-)
>
> Maple/Magma/etc. are not optional packages.

Yup. Like graphviz they are external programs that can, or not, be
provided by the user's system.

> A network service that only binds to localhost doesn't need ssl.
> Also, one can easily (one line) make an *ssh tunnel* to run the
> notebook securely on a remote machine without installing pyopenssl.

Granted, my example is not 100% bullet proof :-) I am just saying
that, most of the time when you use the notebook code, you do need
other stuff which is not necessarily there by default.

> > An answer is that graphviz is often already installed on the system.
> > In which case nothing needs to be done.
>
> I think graphviz has compiled code, whereas dot2tex is python code, so
> graphviz could be hard to install.

Yes, though it's provided precompiled on most OS, so it's not so bad.

> Also relevant is that graphviz has a nasty GPL-incompatible license,
> so we can never ever include it with Sage, whereas dot2tex has a very,
> very standard open license:
>
> https://code.google.com/p/dot2tex/source/browse/LICENSE

+1

Cheers,
Nicolas

[Jeroen Demeyer]

On 05/22/2013 06:58 PM, Nicolas M. Thiery wrote:
> An answer is that graphviz is often already installed on the system.

Really? Are there any Linux distributions which come with graphviz
installed?

[John H Palmieri]

On Wednesday, May 22, 2013 9:58:56 AM UTC-7, Nicolas M. Thiéry wrote:

On Wed, May 22, 2013 at 09:13:09AM -0700, kcrisman wrote:
>      A standard package which is only useful in the presence of an optional
>      package doesn't make sense to me.

It simplifies our users's life, and that is useful! Also it simplifies
*my* life: I am tired, e.g. during Sage Days, of having to explain
about Sage packages to our complete beginners for something that is
often one of the very first feature that they want to play with.

Is graphviz a run-time dependency of dot2tex? That is, can someone install graphviz after building Sage (including dot2tex, if it were a standard spkg) and have it work, or would they need to rebuild dot2tex?
 
--
John

[Nicolas M. Thiery]

Most likely not on a vanilla Linux distribution. But on my average
beginners machine this happens quite often because they (or some other
user for a shared machine) have needed it before.

Cheers,
Nicolas

[Volker Braun]

Isn't graphviz under the EPL? Thats not so bad. And we don't even want to link to it, so its perfectly fine for us to distribute it.

How about a new category of "extra" spkgs that are nice to have but not really required. We could even install them by default, if wanted. Perhaps only if you have internet access during installation. The license requirements could be relaxed, since by this definition we don't link to it. And I would propose that the failure of build/check of an "extra" spkg on a platform would not prevent Sage from installing successfully.

There is quite a list of TeX packages that the pdf docs use but the average TeX installation does't have. 

[William Stein]

On Wed, May 22, 2013 at 3:36 PM, Volker Braun <vbrau...@gmail.com> wrote:
> Isn't graphviz under the EPL? Thats not so bad. And we don't even want to
> link to it, so its perfectly fine for us to distribute it.

Yes, but EPL is GPL incompatible, so it can't be a standard package.
I shouldn't have used the word "nasty" to describe it in my previous
email.

> How about a new category of "extra" spkgs that are nice to have but not
> really required. We could even install them by default, if wanted. Perhaps
> only if you have internet access during installation. The license
> requirements could be relaxed, since by this definition we don't link to it.

Just to be clear -- are you making any proposal about what's in the
binaries that are distributed from sagemath.org, or just what gets
built when one downloads the source? I'm not comfortable including
any GPL-incompatible packages in the Sage binaries from sagemath.org,
but something like you describe at build time seems like it would be
very useful.

> And I would propose that the failure of build/check of an "extra" spkg on a
> platform would not prevent Sage from installing successfully.
>
> There is quite a list of TeX packages that the pdf docs use but the average
> TeX installation does't have.
>

[leif]

Volker Braun wrote:
> How about a new category of "extra" spkgs that are nice to have but not
> really required. We could even install them by default, if wanted.

I'd say that would probably make sense for dot2tex (as suggested in my
other reply), but not really for graphviz.

> Perhaps only if you have internet access during installation.

Hmmm, that was previously considered a no-go (but wouldn't be necessary
for just dot2tex).

> And I would propose that the failure of build/check of an
> "extra" spkg on a platform would not prevent Sage from installing
> successfully.

That certainly makes sense. We could also list at the end of the build
what the user may install in addition (with the distribution's package
manager) to make Sage "fully functional".*

[That would work for e.g. dot2tex, but other packages need their
prerequisites at build time. So those would actually have to get
rebuilt afterwards, which usually doesn't work with bdists.]


-leif

> There is quite a list of TeX packages that the pdf docs use but the
> average TeX installation does't have.

To make dot2tex's test suite pass, I also had to install pdftk.

____________
* If one day Sage gets a proper 'configure', situation will get better.
(I.e., we could recommend to install this and that package /in
advance/, not to mention --with-foo and --without-foo options.)

--
() The ASCII Ribbon Campaign
/\ Help Cure HTML E-Mail

[Julien Puydt]

Le 23/05/2013 00:53, William Stein a �crit :

> Yes, but EPL is GPL incompatible, so it can't be a standard package.

Isn't it a case of confusion between sage-the-software and
sage-the-distribution?

Quite a few distributions ship graphviz and GPL software -- even debian,
which is pretty touchy when it comes to licenses.

Snark on #sagemath

[Volker Braun]

On Wednesday, May 22, 2013 11:53:53 PM UTC+1, William wrote:

Yes, but EPL is GPL incompatible, so it can't be a standard package.

I agree, if just for license clarity: The default Sage tarball should be all GPL (-compatible).

 

Just to be clear -- are you making any proposal about what's in the
binaries that are distributed from sagemath.org, or just what gets
built when one downloads the source?

I was thinking of both. Build from source + internet => build extras, with a separate build target to just build the standard source. I don't see why we shouldn't be able to distribute that as binary tarball either, its perfectly legal to distribute a GPL and a EPL binary in the same .tar.bz as long as they don't link to each other. That's what Linux distributions do, after all. But if somebody feels strongly against it then we can also distribute the binary builds without extras.

 

[Sébastien Labbé]

> Description of dot2tex
> ----------------------

Nicolas, can you describe the developement community which is behind dot2tex. Is it only one guy? Is it a team? How is dot2tex maintained? Is it a package that never needs fixes?

At #13624, I got a problem with it and reported it upstream [2] last Fall. I got a quick acknowledgement and then silence. Finally the bug was fixed in another way by your patch at #13624 which needs review. Moreover, it seems that the bug will be fixed by #14382 which also needs review.

[1] http://trac.sagemath.org/sage_trac/ticket/13624
[2] http://code.google.com/p/dot2tex/issues/detail?id=32
[3] http://trac.sagemath.org/sage_trac/ticket/14382

So in the future, will all the bug related to dot2tex be fixed in Sage like in this example? Will we have problem with the upstream?

Anyway, I really like this dot2tex spkg, I use it a lot. If good warnings are printed if graphviz is needed and not installed, I agree to make it standard. Like we have code in Sage which are based on convert of ffmpeg :

sage: a = animate([sin(x + float(k)) for k in srange(0,2*pi,0.3)],xmin=0, xmax=2*pi, figsize=[2,1])
sage: a.gif?
...
Note: If neither ffmpeg nor ImageMagick is installed, you will get an
error message like this:

        Error: Neither ImageMagick nor ffmpeg appears to be installed. Saving an
        animation to a GIF file or displaying an animation requires one of these
        packages, so please install one of them and try again.

        See www.imagemagick.org and www.ffmpeg.org for more information.

[Nicolas M. Thiery]

Salut S�bastien!

On Thu, May 23, 2013 at 03:14:59AM -0700, S�bastien Labb� wrote:
> Nicolas, can you describe the developement community which is behind
> dot2tex. Is it only one guy? Is it a team? How is dot2tex maintained? Is
> it a package that never needs fixes?

> So in the future, will all the bug related to dot2tex be fixed in Sage
> like in this example? Will we have problem with the upstream?

That's a good question.

dot2tex is developed and maintained by a single person. He used to be
very reactive, but indeed not so much lately. So we should ask Kjell
about his plans for the future of dot2tex. It would indeed be best if
as much as possible was resolved upstream.

That being said, dot2tex is relatively feature complete (we have been
using it intensively and did not post a feature request in a long
time), robust (one bug report per year?), and clear enough (on the
occasions I had to dig in, I could easily find my way).

So, altogether, maintaining dot2tex is probably a question of fixing a
bug or two per year and cutting an official release from time to
time. I am not volunteering to do that, but we could possibly find a
volunteer in case Kjell would want to pass it on.

Cheers,
Nicolas