How to test whether a package is installed?
Simon King
Related with #8523, I'd like to know how one tests whether some
package (here: database_gap) is already installed. I don't care what
exact version it is.
I understood that the script SAGE_ROOT/spkg/standard/newest_version
can be used. I was reading somewhere that this script is supposed to
be called in SAGE_ROOT/spkg. But calling
./newest_version database_gap
fails as follows:
> standard/newest_version database_gap
/bin/ls: cannot access database_gap-*.spkg: No such file or directory
> standard/newest_version optional/database_gap
/bin/ls: cannot access optional/database_gap-*.spkg: No such file or
directory
But:
> ls optional/database_gap-4.4.1*
optional/database_gap-4.4.10.spkg optional/database_gap-4.4.12.spkg
So, I ended up using it directly in SAGE_ROOT/spkg/optional/, which
works:
> ../standard/newest_version database_gap
database_gap-4.4.12
But is this really how it is supposed to work?
Another problem: It may happen that someone installs an optional
package in a way that there is no .spkg file in SAGE_ROOT/spkg/
optional/. Instead, there only is a file in SAGE_ROOT/spkg/
installed/., but this file does not have an .spkg extension.
Shouldn't SAGE_ROOT/spkg/standard/newest_version better look for (non-
spkg) files in SAGE_ROOT/spkg/installed/? Or what else can one do to
test if a package is installed?
Best regards,
Simon
William Stein
> Hi!
>
> Related with #8523, I'd like to know how one tests whether some
> package (here: database_gap) is already installed. I don't care what
> exact version it is.
>
> I understood that the script SAGE_ROOT/spkg/standard/newest_version
> can be used.
No. That script returns the newest version of the spkg that is
*available* to be installed in the users spkg/standard directory.
> I was reading somewhere that this script is supposed to
> be called in SAGE_ROOT/spkg. But calling
> ./newest_version database_gap
> fails as follows:
>
>> standard/newest_version database_gap
> /bin/ls: cannot access database_gap-*.spkg: No such file or directory
>
>> standard/newest_version optional/database_gap
> /bin/ls: cannot access optional/database_gap-*.spkg: No such file or
> directory
>
> But:
>> ls optional/database_gap-4.4.1*
> optional/database_gap-4.4.10.spkg optional/database_gap-4.4.12.spkg
>
> So, I ended up using it directly in SAGE_ROOT/spkg/optional/, which
> works:
>> ../standard/newest_version database_gap
> database_gap-4.4.12
>
> But is this really how it is supposed to work?
The directory SAGE_ROOT/spkg/installed contains a file for each spkg
that was installed, that contains metainformation about when that
package was installed. I think you should look at that. The contents
of spkg/standard and spkg/optional aren't guaranteed to tell you
anything useful at all.
> Another problem: It may happen that someone installs an optional
> package in a way that there is no .spkg file in SAGE_ROOT/spkg/
> optional/. Instead, there only is a file in SAGE_ROOT/spkg/
> installed/., but this file does not have an .spkg extension.
>
> Shouldn't SAGE_ROOT/spkg/standard/newest_version better look for (non-
> spkg) files in SAGE_ROOT/spkg/installed/? Or what else can one do to
> test if a package is installed?
No. newest_version is irrelevant. One should look at spk/installed/
>
> Best regards,
> Simon
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>
> To unsubscribe from this group, send email to sage-devel+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
>
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
Simon King
On 28 Mrz., 19:55, William Stein <wst...@gmail.com> wrote:
> ...
> No. newest_version is irrelevant. One should look at spk/installed/
So, one would test whether
ls spkg/installed/database_gap*
results in an error?
Cheers,
Simon
William Stein
Hmm. I think the best thing is to create a new script in
SAGE_ROOT/local/bin/
and encapsulate the check in there. Then in your script call that.
I.e., make something that does what you thought newest_version did.
First find what the relevant commands are in say apt, rpm, etc., as
inspiration for a command name.
Regarding the implementation of that script, you could maybe check for
ls returning an error (or whatever) as above. However, better
would be to write something in Python, e.g.,
#!/usr/bin/env python
import os
# do something with os.listdir()
---
By factoring out this functionality in a script, we avoid awkward
buggy so called "best practices" appearing in spkg-install scripts.
As much as possible, anything that could possibly be replicated in
several spkg-install scripts should be factored out.
William
>
> Cheers,
Dr. David Kirkby
> Regarding the implementation of that script, you could maybe check for
> ls returning an error (or whatever) as above. However, better
> would be to write something in Python, e.g.,
>
> #!/usr/bin/env python
>
> import os
>
> # do something with os.listdir()
>
> ---
>
> By factoring out this functionality in a script, we avoid awkward
> buggy so called "best practices" appearing in spkg-install scripts.
> As much as possible, anything that could possibly be replicated in
> several spkg-install scripts should be factored out.
>
>
> William
That's not going to work until python is installed. Why not make it a shell
script instead - it would work before python is built, irrespective of whether
someone has python installed on their machine.
Dave
William Stein
<david....@onetel.net> wrote:
> William Stein wrote:
>
>> Regarding the implementation of that script, you could maybe check for
>> ls returning an error (or whatever) as above. However, better
>> would be to write something in Python, e.g.,
>>
>> #!/usr/bin/env python
>>
>> import os
>>
>> # do something with os.listdir()
>>
>> ---
>>
>> By factoring out this functionality in a script, we avoid awkward
>> buggy so called "best practices" appearing in spkg-install scripts.
>> As much as possible, anything that could possibly be replicated in
>> several spkg-install scripts should be factored out.
>>
>>
>> William
>
> That's not going to work until python is installed.
All that matters is that Python is installed system-wide on the
persons computer.
> Why not make it a shell
> script instead - it would work before python is built, irrespective of
> whether someone has python installed on their machine.
I'm fine with requiring some Python be installed as a prerequisite for
building Sage.
Python is a fine language for writing system/build system scripts.
Several of the Sage components use Perl (NTL and Pari), so some
systemwide Perl is a prereq for installing Sage. Requiring some
Python is just as reasonable as requiring some Perl, which we already
do.
-- William
>
> Dave
Simon King
On 29 Mrz., 00:31, "Dr. David Kirkby" <david.kir...@onetel.net> wrote:
> ...
> That's not going to work until python is installed. Why not make it a shell
> script instead - it would work before python is built, irrespective of whether
> someone has python installed on their machine.
For the application that I have in mind -- namely in spkg-install for
optional packages, and perhaps for upgrading (so that one needs to
find out whether one component is outdated) -- it is granted that
python is present, because at that point Sage is already built.
If I understand correctly, the tasks are
- to test whether SAGE_ROOT/spkg/installed/ contains a file whose
name begins with a given string, and
- if there are several such files, to extract and compare the version
numbers and return the file name that belongs to the highest version
number.
I have no idea how to accomplish the second task with a shell script
(I am pretty much of a newbie in shell scripts). And even for the
first task I could imagine that there is a problem of platform
specific shell commands (but you certainly know better than I).
For extacting the version numers, can we assume that any spkg "foo"
will have a name including the version number like foo-
x[.y[.z[...]]].spkg, where x,y,z,... are numbers, except for the last
one, which can be of the form 'p' plus a number?
.
Best regards,
Simon
William Stein
> Hi David,
>
> On 29 Mrz., 00:31, "Dr. David Kirkby" <david.kir...@onetel.net> wrote:
>> ...
>> That's not going to work until python is installed. Why not make it a shell
>> script instead - it would work before python is built, irrespective of whether
>> someone has python installed on their machine.
>
> For the application that I have in mind -- namely in spkg-install for
> optional packages, and perhaps for upgrading (so that one needs to
> find out whether one component is outdated) -- it is granted that
> python is present, because at that point Sage is already built.
>
> If I understand correctly, the tasks are
> - to test whether SAGE_ROOT/spkg/installed/ contains a file whose
> name begins with a given string, and
> - if there are several such files, to extract and compare the version
> numbers and return the file name that belongs to the highest version
> number.
>
> I have no idea how to accomplish the second task with a shell script
> (I am pretty much of a newbie in shell scripts). And even for the
> first task I could imagine that there is a problem of platform
> specific shell commands (but you certainly know better than I).
Moreover, even if you could, a lot of people probably couldn't read the results.
Also, another assumption that is fair to make is that most Sage users
are fluent in Python.
>
> For extacting the version numers, can we assume that any spkg "foo"
> will have a name including the version number like foo-
> x[.y[.z[...]]].spkg, where x,y,z,... are numbers, except for the last
> one, which can be of the form 'p' plus a number?
Is that assumption true of all current standard and optional spkg's?
You can check as easily as I. If so, we could make it a requirement.
However, if we do, then we *must* add logic to verify this to the
"sage -pkg" command, since making it a requirement by putting it on
some wiki page somewhere, won't work.
Dr. David Kirkby
> On Sun, Mar 28, 2010 at 3:31 PM, Dr. David Kirkby
> <david....@onetel.net> wrote:
>> William Stein wrote:
>>
>>> Regarding the implementation of that script, you could maybe check for
>>> ls returning an error (or whatever) as above. However, better
>>> would be to write something in Python, e.g.,
>>>
>>> #!/usr/bin/env python
>>>
>>> import os
>>>
>>> # do something with os.listdir()
>>>
>>> ---
>>>
>>> By factoring out this functionality in a script, we avoid awkward
>>> buggy so called "best practices" appearing in spkg-install scripts.
>>> As much as possible, anything that could possibly be replicated in
>>> several spkg-install scripts should be factored out.
>>>
>>>
>>> William
>> That's not going to work until python is installed.
>
> All that matters is that Python is installed system-wide on the
> persons computer.
It is true that the system needs an install of python, but there may not be one
present. I would assume some of the cut-down distributions of Linux do not
include python. Does Cygwin install it by default?
>> Why not make it a shell
>> script instead - it would work before python is built, irrespective of
>> whether someone has python installed on their machine.
>
> I'm fine with requiring some Python be installed as a prerequisite for
> building Sage.
> Python is a fine language for writing system/build system scripts.
>
> Several of the Sage components use Perl (NTL and Pari), so some
> systemwide Perl is a prereq for installing Sage. Requiring some
> Python is just as reasonable as requiring some Perl, which we already
> do.
Agreed, though I would have thought perl is more common than python. But the use
of any language like that (perl/pyton/ruby ...) seems a bit of a sledgehammer to
crack a walnut. The overhead of calling python must be much greater than that of
the shell. Couple that with the fact you then add another prerequisite, it seems
a bit pointless to me.
Anyway, it's your decision.
Dave
William Stein
<david....@onetel.net> wrote:
> It is true that the system needs an install of python, but there may not be
> one present. I would assume some of the cut-down distributions of Linux do
> not include python. Does Cygwin install it by default?
Building Sage on Cygwin -- when that is supported -- will require the
user to first install a certain list of Cygwin packages. Python is
trivial to install in Cygwin (as one of those packages).
>
>>> Why not make it a shell
>>> script instead - it would work before python is built, irrespective of
>>> whether someone has python installed on their machine.
>>
>> I'm fine with requiring some Python be installed as a prerequisite for
>> building Sage.
>> Python is a fine language for writing system/build system scripts.
>>
>> Several of the Sage components use Perl (NTL and Pari), so some
>> systemwide Perl is a prereq for installing Sage. Requiring some
>> Python is just as reasonable as requiring some Perl, which we already
>> do.
>
> Agreed, though I would have thought perl is more common than python. But the
> use of any language like that (perl/pyton/ruby ...) seems a bit of a
> sledgehammer to crack a walnut.
Python is not a sledgehammer; it is an elegant tool, which can be used
to produces beautiful readable maintainable and portable code.
> Anyway, it's your decision.
I'm for allowing the use of Python in the Sage build system.
-- William
Georg S. Weber
> Python is not a sledgehammer; it is an elegant tool, which can be used
> to produces beautiful readable maintainable and portable code.
>
> > Anyway, it's your decision.
>
> I'm for allowing the use of Python in the Sage build system.
>
> -- William
Under $SAGRE_ROOT/local/bin/, several of the Sage infrastructure
scripts are already written in Python, instead of "bash" (except for
the very first she-bang line "#!/usr/bin/env python") --- see e.g.
"sage-clone".
For what it's worth, I really like the idea to refactor/rewrite the
Sage build system/infrastructure more or less completely in Python. Oh
well, if I had time for that, I'd rather devote it to help the release
managers.
Cheers,
Georg
Dr. David Kirkby
On Mar 29, 8:01 pm, "Georg S. Weber" <georgswe...@googlemail.com>
wrote:
I see nothing wrong with that as such. What I do find odd is to make
Python a prerequisite, then also include the python sources too. That
means people are going to need to have two copies of python. I would
imagine such a strategy would be even more difficult to "sell" to
groups like Debian for the perposes of packaging. One could argue a
requirement to ship python sources, but it gets even more difficult to
argue that python is a prerequisite, but we still ship the sources.
Personally I would think that anything that needs to be done before
python is built should be done in bash scripts. Of course, there is
nothing stopping python being built very early on.
Dave
Nils Bruin
> I'm for allowing the use of Python in the Sage build system.
It seems to me that with a little care one could have python used in
some parts of the build system and avoid declaring python a
prerequisite for python.
If a package "requires" another spkg, I assume it only gets built
after the the requirement has successfully built. In that case, any
package that wants to use python during build time just has python as
a requirement. I'd expect that python can be one of the first things
that gets built anyway.
Dr. David Kirkby
> On Mar 28, 9:10 pm, William Stein <wst...@gmail.com> wrote:
>
>> I'm for allowing the use of Python in the Sage build system.
>
> It seems to me that with a little care one could have python used in
> some parts of the build system and avoid declaring python a
> prerequisite for python.
Yes, I think so too.
> If a package "requires" another spkg, I assume it only gets built
> after the the requirement has successfully built. In that case, any
> package that wants to use python during build time just has python as
> a requirement. I'd expect that python can be one of the first things
> that gets built anyway.
Currently the python built in Sage requires zlib, termcap, sqlite, libpng etc
all to be built first, so it's not quite as easy as that.
A sensible strategy might be:
1) Check if there is python on the system. If there is no python, go to stage 2.
If there is a usable pyton, go to stage 3.
2) Build a very basic python, without any extra libraries.
3) Build zlib, termcap sqlite, libpng etc.
4) Build a more complete python.
That avoids python being a perquisite. It just seems wrong to me to make a
package a perquisite, whilst shipping the sources for that package. Personally
though, if one is only testing if a file exists or not, a /bin/sh script would
incur less overhead.
(I'be made the assumption that you don't actually need zlib, sqlite etc to
create a basic pyton. That might not be true in the case of zlib).
Dave
William Stein
<david....@onetel.net> wrote:
> Nils Bruin wrote:
>>
>> On Mar 28, 9:10 pm, William Stein <wst...@gmail.com> wrote:
>>
>>> I'm for allowing the use of Python in the Sage build system.
>>
>> It seems to me that with a little care one could have python used in
>> some parts of the build system and avoid declaring python a
>> prerequisite for python.
>
> Yes, I think so too.
>
>> If a package "requires" another spkg, I assume it only gets built
>> after the the requirement has successfully built. In that case, any
>> package that wants to use python during build time just has python as
>> a requirement. I'd expect that python can be one of the first things
>> that gets built anyway.
>
>
> Currently the python built in Sage requires zlib, termcap, sqlite, libpng
> etc all to be built first, so it's not quite as easy as that.
>
> A sensible strategy might be:
>
> 1) Check if there is python on the system. If there is no python, go to
> stage 2. If there is a usable pyton, go to stage 3.
Let's make some system-wide Python a prerequisite. Full stop. I
think all of our officially supported platforms come with Python by
default, anyways.
William
Dima Pasechnik
On Mar 30, 3:21 am, "Dr. David Kirkby" <david.kir...@onetel.net>
wrote:
[...]
> What I do find odd is to make
> Python a prerequisite, then also include the python sources too. That
> means people are going to need to have two copies of python.
most likely, two different versions.
E.g. Debian stable does not have Python 2.6, at all.
It is in Debian testing, but not as a default Python.
(Most of Linux systems out there have a bunch of different versions of
Python
installed, IMHO)
Cygwin does not have Python 2.6 at present, either.
> I would
> imagine such a strategy would be even more difficult to "sell" to
> groups like Debian for the perposes of packaging. One could argue a
> requirement to ship python sources, but it gets even more difficult to
> argue that python is a prerequisite, but we still ship the sources.
> Personally I would think that anything that needs to be done before
> python is built should be done in bash scripts.
really, "some" Python 2.5 or 2.4 or 2.6 is quite easy to get for any
system
Sage supports or intends to support.
Getting exactly the same Python as Sage runs on is quite different
story, and can be hard.
The only alternative would be to make Sage running on the zillon
Pythons 2.x.y , for each x.y>x0.y0, out
there, and this seems infeasible.
Dima
David Kirkby
To build on Solaris 10 (or OpenSolaris) it would need to work with python 2.4.4.
drkirkby@kestrel:~$ python
Python 2.4.4 (#1, Jan 10 2007, 01:25:01) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
I can see there might be even more Solaris issues!
Since python 3 is not backward compatible with python 2, there could
be even more problems in future.
Personally I would have stuck to POSIX scripts until python in Sage was built.
Dave
Simon King
On Mar 30, 2:50 pm, David Kirkby <david.kir...@onetel.net> wrote:
> ...
> I can see there might be even more Solaris issues!
>
> Since python 3 is not backward compatible with python 2, there could
> be even more problems in future.
Probably one could write a simple Python script that is both
executable in Python 2 and Python 3.
However, I share your general concerns.
> Personally I would have stuck to POSIX scripts until python in Sage was built.
Yes.
Is it true that Python is the first thing that is built when
installing Sage?
If it is, then IMO there is no problem at all in using Python scripts.
The spkg-install of the python spkg must not rely on Python, but
everything else can use Python.
Cheers,
Simon
William Stein
> Hi Dave,
>
> On Mar 30, 2:50 pm, David Kirkby <david.kir...@onetel.net> wrote:
>> ...
>> I can see there might be even more Solaris issues!
>>
>> Since python 3 is not backward compatible with python 2, there could
>> be even more problems in future.
>
> Probably one could write a simple Python script that is both
> executable in Python 2 and Python 3.
> However, I share your general concerns.
For any build system stuff, I think it is trivial to write scripts
that work in python 2.3, 2.4, 2.5, 2.6, 3.x, etc.
Just don't use decorators, class properties, and use print as a function call.
>> Personally I would have stuck to POSIX scripts until python in Sage was built.
>
> Yes.
No.
> Is it true that Python is the first thing that is built when
> installing Sage?
No, certainly not.
> If it is, then IMO there is no problem at all in using Python scripts.
> The spkg-install of the python spkg must not rely on Python, but
> everything else can use Python.
Python isn't built first.
However, there is still no problem using python in the build system.
As was pointed out above, we already do. I would like to encourage
more use of Python.
William
Robert Bradshaw
sage: is_package_installed('python')
True
On Mar 30, 2010, at 10:02 AM, William Stein wrote:
> On Tue, Mar 30, 2010 at 8:04 AM, Simon King
> <simon...@nuigalway.ie> wrote:
>> Hi Dave,
>>
>> On Mar 30, 2:50 pm, David Kirkby <david.kir...@onetel.net> wrote:
>>> ...
>>> I can see there might be even more Solaris issues!
>>>
>>> Since python 3 is not backward compatible with python 2, there could
>>> be even more problems in future.
>>
>> Probably one could write a simple Python script that is both
>> executable in Python 2 and Python 3.
>> However, I share your general concerns.
>
> For any build system stuff, I think it is trivial to write scripts
> that work in python 2.3, 2.4, 2.5, 2.6, 3.x, etc.
> Just don't use decorators, class properties, and use print as a
> function call.
In particular, it's much, much easier than writing (let alone
maintaining) cross platform shell scripts.
>>> Personally I would have stuck to POSIX scripts until python in
>>> Sage was built.
>>
>> Yes.
>
> No.
>
>> Is it true that Python is the first thing that is built when
>> installing Sage?
>
> No, certainly not.
>
>> If it is, then IMO there is no problem at all in using Python
>> scripts.
>> The spkg-install of the python spkg must not rely on Python, but
>> everything else can use Python.
>
> Python isn't built first.
Just as an interesting aside, Python compiles the minimal amount it
needs to run from C, then uses Python to orchestrate the rest of its
own build process.
> However, there is still no problem using python in the build system.
> As was pointed out above, we already do. I would like to encourage
> more use of Python.
+1. I would guess the fact that the build system is written in a
(relative to Python) brittle and arcane language is one of the reasons
so few people dare touch or fix it.
- Robert
Peter Jeremy
>Let's make some system-wide Python a prerequisite. Full stop.
It's not quite 1st April even here.
I could understand Python being a prerequisite if Sage was going to
use it at runtime but requiring it to be present solely to build
another copy of Python strikes me as wasteful. If Sage is aiming at
being self-contained, it needs to reduce its reliance on
prerequisites, not increase it. What is wrong with using shell
scripts to build Python?
If a user wants to download Sage as binary package, will they still
need Python as a prerequisite?
> I
>think all of our officially supported platforms come with Python by
>default, anyways.
There are Python packages available for most current operating
systems. There are probably far fewer operating system distributions
that have Python installed by default. I suspect this decision will
only increase the effort required to port Sage to other platforms.
--
Peter Jeremy
William Stein
> On 2010-Mar-29 16:08:48 -0700, William Stein <wst...@gmail.com> wrote:
>>Let's make some system-wide Python a prerequisite. Full stop.
>
> It's not quite 1st April even here.
This is not an April fools joke.
>
> I could understand Python being a prerequisite if Sage was going to
> use it at runtime but requiring it to be present solely to build
> another copy of Python strikes me as wasteful. If Sage is aiming at
> being self-contained, it needs to reduce its reliance on
> prerequisites, not increase it. What is wrong with using shell
> scripts to build Python?
In some cases it sucks. I could go on and on, but won't, since this issue has
already been addressed above.
> If a user wants to download Sage as binary package, will they still
> need Python as a prerequisite?
No, of course not -- Python comes with the Sage binary !?
>
>> I
>>think all of our officially supported platforms come with Python by
>>default, anyways.
>
> There are Python packages available for most current operating
> systems. There are probably far fewer operating system distributions
> that have Python installed by default. I suspect this decision will
> only increase the effort required to port Sage to other platforms.
Is there even *one* operating system that building Sage from source
officially supports that doesn't
include Python by default or with their build-essentials package
(which is needed to build sage)? I've asked this multiple times in
this thread now, and nobody has replied.
Here is the list of supported platforms, from the README.txt:
PROCESSOR OPERATING SYSTEM
x86 32-bit Linux -- Debian, Ubuntu, CentOS (=Red Hat),
Fedora, openSUSE, Mandriva
x86_64 64-bit Linux -- Debian, Ubuntu, CentOS (=Red Hat),
Fedora, openSUSE, Mandriva
IA-64 Itanium 2 64-bit Linux -- Red Hat, SUSE
x86 Apple Mac OS X 10.5.x
PPC Apple Mac OS X 10.5.x
Use Sage on Microsoft Windows via VirtualBox. We do not always test on
OS X 10.4, but Sage should work there fine.
---
In each case above, the Linux distro is meant to be the latest stable
released version.
-- William