pylab.org domain

[Thomas Kluyver]

Hi all,

Charles Vejnar, the owner of the pylab.org domain, is willing to transfer it to us for the unified scipy branding exercise. Is NumFOCUS the best organisation to hold the domain name? What details do we need to give him so he can make the transfer?

Thanks,
Thomas

[Fernando Perez]

I would say yes, that numfocus should be the one holding things like
this. That's part of the point of having a neutral (with regards to
the individual projects) entity like numfocus, and it's precisely the
role the PSF plays for python itself.

I'll let Leah and/or Anthony comment on the administrative specifics;
not having done this yet I'm not sure what's involved.

Cheers,

f

[Anthony Scopatz]

Hello Thomas, Fernando, 

I agree that this would be well within our purview to have.  I believe that there are other domains that we have/are interested in acquiring.  

In terms of administration, I would be happy to handle all of the transactions required.  However, any domain management I think should be done by someone else.  We will have to bring it up with the board, but If there is someone interested in volunteering to help here, that would be great.  

Be Well

Anthony

[Thomas Kluyver]

Thanks Fernando, Anthony,

I can set up the DNS records - is there anything else involved in
managing a domain?

Thomas

[Anthony Scopatz]

On Tue, Sep 11, 2012 at 11:18 AM, Thomas Kluyver <tak...@gmail.com> wrote:

Thanks Fernando, Anthony,

I can set up the DNS records - is there anything else involved in
managing a domain?

That is pretty much it for a single domain.  But if we are talking about organizing multiple domains it would be good to have someone who keeps a record of everyone that we have, manages requests for changing them, etc. 

[Thomas Kluyver]

On 11 September 2012 17:32, Anthony Scopatz <sco...@gmail.com> wrote:
> That is pretty much it for a single domain. But if we are talking about
> organizing multiple domains it would be good to have someone who keeps a
> record of everyone that we have, manages requests for changing them, etc.

OK, that makes sense. That's not something I'd want to take on myself, though.

I'll ask Charles for an email address so we can bring him into the
larger discussion.

Thanks,
Thomas

[Anthony Scopatz]

On Tue, Sep 11, 2012 at 11:41 AM, Thomas Kluyver <tak...@gmail.com> wrote:

On 11 September 2012 17:32, Anthony Scopatz <sco...@gmail.com> wrote:
> That is pretty much it for a single domain.  But if we are talking about
> organizing multiple domains it would be good to have someone who keeps a
> record of everyone that we have, manages requests for changing them, etc.

OK, that makes sense. That's not something I'd want to take on myself, though.

Sure, this was more a request for the community at large.

[Travis Oliphant]

Hey Thomas, 

Thanks for coordinating this.     NumFOCUS has the assistance of Continuum's IT staff whenever necessary and so we can manage this. 

The current care-taker  of the numpy.org domain has also agreed to have NumFOCUS listed as the registrar, so both pylab.org and numpy.org can be managed by NumFOCUS. 

Best,

-Travis

[Charles Vejnar]

Hi all,

Sorry for the long delay. Quite busy at work.

I am the owner of the pylab.org domain.

As I explained to Thomas, I bought the domain to protect it, and had the project to share some of my experience with Pylab (as I am using nympy/scipy/matplotlib for my work), but never really had the time to do it.

So I am willing to transfer the domain, and it seems the Numfocus foundation is a good place for that. I would like however to have some details of what you want to do with it.

Regards,

Charles

[Thomas Kluyver]

Thanks Charles,

On 15 September 2012 10:08, Charles Vejnar <charles...@gmail.com> wrote:
> So I am willing to transfer the domain, and it seems the Numfocus foundation
> is a good place for that. I would like however to have some details of what
> you want to do with it.

I'll describe my vision; several people have had similar ideas, so
others might have a slightly different take on it, but I think we
agree on the main points.

As you know, Python, numpy, scipy and matplotlib are a powerful set of
tools for scientific computing, along with many more projects that
build on that foundation (I'm involved in developing IPython, which we
consider a key part of this ecosystem). But when someone is evaluating
Python and R, or Python and Matlab, where do they start reading?
- Searching for Python leads to a general purpose programming
language, which can't even do plotting out of the box.
- The scipy.org website makes some effort to be a general portal for
scientific computing in Python, but the name scipy also refers to a
package, and the download page is for packages, not a complete
environment.
- Distributions like EPD and Python(x,y) offer a one-stop-shop, but a
new user is unlikely to search for those unless they already know
about them.

So the idea is to develop Pylab as an umbrella term for
Python+numpy+scipy+matplotlib and the ecosystem of packages building
on that. Then pylab.org will offer an introduction, links to relevant
documentation, and a download page which points to inclusive
distributions (as well as offering instructions for installing the
necessary components manually). I also envision that Pylab will be a
versioned standard, something like Linux Standard Base, so that
distributions that want to say they include Pylab 2.0 will have to
offer at least Python 2.6, Numpy version n, and so on.

To summarise, I view it as three components: the name, the website and
the standard.

Best wishes,
Thomas

[Charles Vejnar]

Hi,

Well, I share your vision 100%.

So, if this vision is shared by the Numfocus board, then we can talk about the technical details for the transfer.

Regards,

Charles

[Travis Oliphant]

I share this vision as one member of the board.     Jarrod, Fernando, Perry, and our Treasurer Anthony should also chime in.   

-Travis

[Anthony Scopatz]

I am +1 on anything that makes packaging easier.

[Thomas Kluyver]

On 16 September 2012 23:55, Anthony Scopatz <sco...@gmail.com> wrote:
> I am +1 on anything that makes packaging easier.

To be clear, I don't think my proposal really relates to packaging, as
in producing distributable packages of software. I'm aiming to solve
user and documentation problems with this, not technical problems.

Best wishes,
Thomas

[Anthony Scopatz]

Sure, I understand that.  But part of the problem is that people don't know where to go to get X piece of software.  Even if pylab.org is just a list of links for different code sources I think that addresses some of the ecosystem problems we have now.  I didn't mean to suggest that this would solve the packaging problem.

 

Best wishes,
Thomas

[Scott Collis]

Exactly (not as a board member ;) but as a user!)

No insult meant to Enthought but scipy.org is becoming less and less usable. A one stop shop with cookbooks, tutorials, links and perhaps some web2 apps (discussion forum, wiki?) would be a useful resource and I would certainly send students and colleagues in that direction..

Pylab is a nice address... What is the history behind importing matplotlib as pylab? (I don't know how I arrived at this.. But I always do an import pylab... pylab.plot etc..)

Sent from an iPad, please forgive my fat fingers...

----
Dr Scott Collis
ARM Precipitation Radar Translator

ARM Climate Research Facility
Environmental Science Division
Argonne National Laboratory
Cell: +1 630 235 8025
Office: +1
630 252 0550
http://radar.arm.gov

[Thomas Kluyver]

On 17 September 2012 00:07, Scott Collis <scolli...@gmail.com> wrote:
> No insult meant to Enthought but scipy.org is becoming less and less usable. A one stop shop with cookbooks, tutorials, links and perhaps some web2 apps (discussion forum, wiki?) would be a useful resource and I would certainly send students and colleagues in that direction..

For the moment, that's beyond the scope of what I'm aiming for. I want
to just make a starting point which introduces the pylab stack and
points you to ways to install it. Integrating and improving
documentation is important, but it's a job for another day.

> Pylab is a nice address... What is the history behind importing matplotlib as pylab? (I don't know how I arrived at this.. But I always do an import pylab... pylab.plot etc..)

The original idea was to provide a system familiar to Matlab users -
if you do 'from pylab import *', the commands are similar to Matlab,
with a flattened namespace of many useful functions. It's good for
interactive use, although for lasting code a more Pythonic approach is
preferable. One concern about using 'pylab' as the name is that it
sounds like a Matlab knock-off, but I think it has now got a
reputation of its own.

Thanks,
Thomas

[Benjamin Root]

On Sunday, September 16, 2012, Scott Collis wrote:

Exactly (not as a board member ;) but as a user!)

No insult meant to Enthought but scipy.org is becoming less and less usable. A one stop shop with cookbooks, tutorials, links and perhaps some web2 apps (discussion forum, wiki?) would be a useful resource and I would certainly send students and colleagues in that direction..

Pylab is a nice address... What is the history behind importing matplotlib as pylab? (I don't know how I arrived at this.. But I always do an import pylab... pylab.plot etc..)

from pylab import *

It imports the pyplot and numpy namespaces into the current namespace to provide an interface familiar to Matlab users.  If you are calling pylab.plot, you might as well to the accepted:

import matplotlib.pyplot as plt

And do: plt.plot().  The pylab mode was intended to be the bridge used to help matlab and other users to convert.  When they become comfortable with that, and want more advanced features, we move them off the pylab mode over to pyplot.

Cheers!

Ben Root

[Eric Firing]

On 2012/09/16 1:07 PM, Scott Collis wrote:
> Pylab is a nice address... What is the history behind importing
> matplotlib as pylab? (I don't know how I arrived at this.. But I
> always do an import pylab... pylab.plot etc..)

The pylab module in matplotlib dates back to Dec. 9, 2004:

http://currents.soest.hawaii.edu/hgstage/mpl_from_git/rev/e2de80ab218c

Eric

[Scott Collis]

Thankyou all for the enlightenment!

Sent from an iPad, please forgive my fat fingers...

----

[Dag Sverre Seljebotn]

On 09/17/2012 01:04 AM, Anthony Scopatz wrote:
> On Sun, Sep 16, 2012 at 6:00 PM, Thomas Kluyver <tak...@gmail.com
> <mailto:tak...@gmail.com>> wrote:
>
> On 16 September 2012 23:55, Anthony Scopatz <sco...@gmail.com

> <mailto:sco...@gmail.com>> wrote:
> > I am +1 on anything that makes packaging easier.
>
> To be clear, I don't think my proposal really relates to packaging, as
> in producing distributable packages of software. I'm aiming to solve
> user and documentation problems with this, not technical problems.
>
>
> Sure, I understand that. But part of the problem is that people don't
> know where to go to get X piece of software. Even if pylab.org

> <http://pylab.org> is just a list of links for different code sources I

> think that addresses some of the ecosystem problems we have now. I
> didn't mean to suggest that this would solve the packaging problem.

I feel this is missing the point. The current scipy.org aims to fix this
too, it's just that it is currently poorly executed.

Whatever domain name is chosen, the website on it must be well executed,
and better than the current scipy.org. I think everybody agrees on that.

I feel this is strictly a question of names; i.e. it is proposed that
"pylab" is the name one wants to use to market "the stack" (related to
the "Sciome" discussion Travis kicked off.)

(E.g., I still think it would be better to use scipy.org and the SciPy
name for this; in anticipation of in a year or so break up SciPy into
individual scikits, to avoid having yet another name floating around. If
we use "Pylab", then one should also push for "The SciPy conference" to
be named "The Pylab conference" and so on and make SciPy strictly about
the library. But since that library is just a collection of unrelated
pieces I'd personally rather have the library yield. Because that's just
because I almost never use "import scipy" myself.

Anyway, given the recent thread on the SciPy list I'm in the minority
here; I just wanted to point out that that is what this discussion is
*about*.)

Dag

[Thomas Kluyver]

On 17 September 2012 09:03, Dag Sverre Seljebotn

<d.s.se...@astro.uio.no> wrote:
> (E.g., I still think it would be better to use scipy.org and the SciPy name
> for this; in anticipation of in a year or so break up SciPy into individual
> scikits, to avoid having yet another name floating around. If we use
> "Pylab", then one should also push for "The SciPy conference" to be named
> "The Pylab conference" and so on and make SciPy strictly about the library.
> But since that library is just a collection of unrelated pieces I'd
> personally rather have the library yield. Because that's just because I
> almost never use "import scipy" myself.

SciPy is in a sense the name we want, but I think it would be very
disruptive to rename or break up scipy-the-package. And I don't think
trying to share the name between the package and the stack can work
well. So pylab is a compromise - it's a familiar name that's already
used in a similar way, and can be reused without much disruption.

The various scipy conferences may well keep their names for some time
- by the time people are familiar enough with the tools to go to
conferences, they can put up with a name that's different for
historical reasons. But if this proposal works, then in a few years
things like the conferences and scipy-central might be keen to look at
using the name pylab.

Best wishes,
Thomas

[Chris Kees]

While the name is certainly important, I don't really have a preference. The notion of a versioned stack is one that we have been working on for a while now, so I really support what you're trying to do here.

The stack specification we put together from various DoD Python users working on HPC applications is still up here: http://fixingscientificsoftwaredistribution.blogspot.com.  It's obviously too large and specialized for the basic pylab(scipy or whatever name you guys choose), but it might be worth paying attention to.  The Python+numpy+scipy+IPython stack is simply not complete enough to be of any use to me and many developers that I've discussed this issue with.  The group inside the DoD had discussed a simple decomposition into several levels. For example if the stack is called pylab then maybe something pylab-lite, pylab-medium, pylab-full. Coupled with a testing framework, this would allow those of us working on managed machines to start using the stack provided by the administrators rather than building the whole stack ourselves with ad hoc distribution utilities.  I want to say to the system maintainers "the new machine (system upgrade, etc.)  must run pylab-medium 1.1 or greater on the compute nodes and pylab-full on the login nodes", and I want to be able to verify it  easily with something like 'python pylab-medium-test.py'.

Anyway, I'm hoping this numfocus effort leads to a versioned stack that can get me within 80% of the stack I actually need. The hashdist tools that Ondrej and Dag will hopefully implement in the coming months would then give me the tools I need to manage the full stack that my group needs.

Chris

[Thomas Kluyver]

Thanks Chris,

On 18 September 2012 18:49, Chris Kees <cek...@gmail.com> wrote:
> It's obviously too large and specialized for the basic pylab(scipy or
> whatever name you guys choose), but it might be worth paying attention to.
> The Python+numpy+scipy+IPython stack is simply not complete enough to be of
> any use to me and many developers that I've discussed this issue with.

The idea is certainly that this should be a base, not a comprehensive
stack. As the blog you linked to also mentions, numpy and matplotlib
are some of the more complex bits to compile, so having those
foundations consistently available should already be valuable.

I also intend the standard to say that distributions advertising
themselves as pylab compliant should provide an easy way to install
arbitrary Python packages into the distribution.

> The
> group inside the DoD had discussed a simple decomposition into several
> levels. For example if the stack is called pylab then maybe something
> pylab-lite, pylab-medium, pylab-full.

That's an interesting idea. Similarly, in Ubuntu, you can install R as
r-base-core or r-recommended, which includes a larger selection of
packages. I think there are similar things for TeX packages, too. If
we did go down this route, I'd favour names like base/core,
recommended, extras. But, at least at first, I think we should keep it
simple and define just one set of packages. I'll bring that back to
broader discussion on the Scipy-user list.

Best wishes,
Thomas

[Travis Oliphant]

This is an interesting idea. Having a few levels of pylab packages could be definitely useful and something that this list and wider community could participate in defining. Having pylab, pylab-full as a start (I'm happy to ensure that AnacondaCE, which is an attribution-only binary build of something like pylab-full could be available as a reference implementation of the pylab-full specification).

After getting these communicated, different levels could be created later.

-Travis





>
> Best wishes,
> Thomas

[David Warde-Farley]

On Tue, Sep 18, 2012 at 4:25 PM, Travis Oliphant <teoli...@gmail.com> wrote:

> This is an interesting idea. Having a few levels of pylab packages could be definitely useful and something that this list and wider community could participate in defining.

Just so I'm on the same page, we're talking about essentially
"metapackages" that get bundled together in binary form, right?

For whatever it's worth, I would urge great caution in efforts to
rename either SciPy-the-package or SciPy-the-conference, especially
the latter where a) SciPy is merely an abbreviation, and b)
conferences thrive on momentum and require it to attract both
attendees and submissions. Losing hard-earned name recognition (or
worse, swapping names such that SciPy becomes further overloaded)
would be a bad thing, in my humble opinion.

I think the "pylab" notion is interesting, however, and the legacy
that the name has means that it will fit right in with the way current
users interact with at least the NumPy/IPython/matplotlib trio.

David

[Thomas Kluyver]

To get this thread back to its original topic: Jarrod, Fernando, Perry, are you broadly behind what I'm trying to build around the name Pylab?

Fernando, from discussion on scipy-user, I think you agree with me on the overall idea, although we've disagreed on the nature of what the specification should aim for.

Best wishes,
Thomas

[Fernando Perez]

On Sat, Sep 22, 2012 at 1:46 PM, Thomas Kluyver <tak...@gmail.com> wrote:
> To get this thread back to its original topic: Jarrod, Fernando, Perry, are
> you broadly behind what I'm trying to build around the name Pylab?
>
> Fernando, from discussion on scipy-user, I think you agree with me on the
> overall idea, although we've disagreed on the nature of what the
> specification should aim for.

Yes, in fact I had started a very similar thread a few months ago that
fizzled out, partly perhaps because I didn't have the bandwidth to
sustain the discussion. You've done a much better job of keeping on
it (I don't recall on which of our myriad lists that thread took
place).

I'm not thrilled with the name, but I've already started using it as I
don't think there's any point in beating that horse any further. I
just noticed you sorted out the github org and that I'm even on it.
Great :)

This idea started to really grate me earlier next year, and while at
strata and pydata I had long conversations with Travis about it. So
by and large I'm +1 on it. One of the things that I hope will over
time grow out of this is a set of tools and specifications that will
enable really smooth integration between 'pylab tools' without the
need for all-out bundling. The individuality that our projects
maintain is, I think, a major asset of our community and it ensures
each sub-project can have a healthy and vibrant dynamic. But we must
present to users a picture that provides a smoother experience on a
number of fronts:

- initial experience on the web. I'm glad there's a plan to
update/improve the scipy.org site, but pylab may serve as an even
better first-landing domain to guide new users through the various
resources and options.
- system-wide initial installations (whether on one machine or many),
Finding the various distribution options, etc.
- package installation (with and without administrative privileges).
- finding help, examples and documentation when confronted with a question.
- sharing of snippets and code. I hope we'll be able to breathe new
life into scipy-central over time

I see this effort as the way to bring some coordination that could
dramatically improve the experience of using python for scientific
work, without introducing an undue burden of coupling between the
projects that make up the ecosystem, forcing them to lose their
individuality, credit and freedom to innovate, or stifling the
possibility of competition and new tools to flourish

A big thanks to you for leading this with the sustained attention it
requires.It will likely be a long-term effort, but I hope that from
the numfocus side we'll be able to also help with some resources over
time. I'm also encouraged by the fact that many new faces have
pitched in on this question, as we need new blood and this is a
potentially very high impact effort.

Cheers,

f

[Thomas Kluyver]

On 22 September 2012 22:10, Fernando Perez <fpere...@gmail.com> wrote:
> I'm not thrilled with the name, but I've already started using it as I
> don't think there's any point in beating that horse any further. I
> just noticed you sorted out the github org and that I'm even on it.
> Great :)

Kudos to Travis for sorting that out, actually - he made the
organisation a few days before I thought of it.

As an aside, I find it odd that Github defaults to concealing the
members of an organisation. I can see cases where you might want to
conceal your membership, but they seem rare enough that it could be
opt-in. At present, people looking at the 'pylab' org will only see
Fernando and I. Travis, Anthony, Jarrod, Perry, you are all members,
but hidden from public view unless you choose otherwise:

https://github.com/pylab?tab=members

> - sharing of snippets and code. I hope we'll be able to breathe new
> life into scipy-central over time

This is also something I've been thinking about, particularly
integrating the notebook. I'll try to get in touch with the person
behind scipy-central in a separate email with ipython-dev.

Thanks,
Thomas

[Jarrod Millman]

Hi Thomas,

Just chiming in to say that in general I am very happy to see an
effort like this moving forward. I've been pretty busy this semester
and haven't had time to closely follow the conversation, so I may be
asking questions or making comments that have been covered in detail
already (in which case sorry for not being more on top of this).

Have you started a draft proposal somewhere (so it can be easily
accessed in its most recent incarnation)? If so, it would be great to
add it to the Pylab github site. If not, I would encourage you to
quickly put up a draft since I believe there is enough consensus
around the general approach to be worth your time and effort in
writing up something semi-formal.

Once you have a draft proposal, it would be great to start figuring
out which pieces can be moved on immediately and which need more
involved discussion. Since we are a community of volunteers and users
with large personal investments in the project and community, I would
urge you to continue being as inclusive and consensus seeking as
possible. Certain aspects of the proposal may warrant discussions on
several project mailing lists and will certainly need to be at least
brought up on them, since many of us are only able to follow a few
lists in any detail.

Are you asking for anything more specific than feedback from us at this point?

I am very happy that you are pushing this vision. It has been voiced
to some extent by many of us and I am sure it is largely shared by
most; but given the distributed nature of our responsibilities and the
fact that there hasn't been clear lines of authority, the vision has
languished waiting for someone to make it into a reality. Thanks for
determinedly pursuing this and I hope you will continue to do so.
From my position in Numfocus I hope that we are able to help you
continuing moving this forward.

Best,
Jarrod

PS. I have a slight preference for retaining the SciPy name for this
effort given that it is such a well-known brand. But I think the
general problem you are trying to address is much more important than
the brand name choice, so please don't let this comment delay you.
Oddly, I just tried to connect to the www.scipy.org and got an error;
I was just going to check to see if it was pointing to
http://scipy.github.com/ yet. Hopefully, that site will move forward
as well.

[Thomas Kluyver]

Thanks for all your comments, Jarrod.

On 22 September 2012 23:03, Jarrod Millman <jarrod....@gmail.com> wrote:
> Have you started a draft proposal somewhere (so it can be easily
> accessed in its most recent incarnation)? If so, it would be great to
> add it to the Pylab github site. If not, I would encourage you to
> quickly put up a draft since I believe there is enough consensus
> around the general approach to be worth your time and effort in
> writing up something semi-formal.

I've not yet put it up in an updatable form, but this post from
yesterday lists the set of packages we were discussing:

http://article.gmane.org/gmane.comp.python.scientific.user/32666

I don't think anyone has argued that anything should be removed from
that list - discussion continues about adding the IPython notebook and
h5py. I'll get round to putting it up as a draft proposal soon.

> Are you asking for anything more specific than feedback from us at this point?

No, just feedback at the moment. I've described before how I see this
as three parts: a name, a standard and a website. Hopefully we have
the first, we're working on the second, and the third I'll build the
first version of myself. There are lots of great ideas about
conventions, documentation and tools that the proposal is already
bringing up (or bringing back). I'm very keen not to get bogged down
with all those discussions at the moment, but I hope the existence of
a unified Pylab will encourage people to work on them.

> Oddly, I just tried to connect to the www.scipy.org and got an error;

That site goes down all too often. I suspect it's not quite annoying
enough for anyone to kick up a big fuss about it, but it was one of
the reasons we moved IPython's website away from scipy.org.

Best wishes,
Thomas

[Jason Grout]

On 9/22/12 5:03 PM, Jarrod Millman wrote:
> PS. I have a slight preference for retaining the SciPy name for this
> effort given that it is such a well-known brand. But I think the
> general problem you are trying to address is much more important than
> the brand name choice, so please don't let this comment delay you.

> Oddly, I just tried to connect to thewww.scipy.org and got an error;

> I was just going to check to see if it was pointing to
> http://scipy.github.com/ yet. Hopefully, that site will move forward
> as well.

It seems to me that it is nice to distinguish between the broader scipy
community and the pylab software suite. In a sense, scipy encompasses
pylab (the lean standard) and also all those extra addons that wouldn't
be in the common pylab toolset.

Thanks,

Jason
--
Jason Grout

[Thomas Kluyver]

On 23 September 2012 04:14, Jason Grout <jason...@creativetrax.com> wrote:
> It seems to me that it is nice to distinguish between the broader scipy
> community and the pylab software suite. In a sense, scipy encompasses pylab
> (the lean standard) and also all those extra addons that wouldn't be in the
> common pylab toolset.

No, I intend the name Pylab to encompass a sense of the community, and
the packages beyond the specification. The ambiguity about
scipy-the-package and scipy-the-ecosystem is precisely what I'm trying
to address.

By analogy with Python itself: you download Python and you get the
standard library installed. But if you ask a question about 'how to do
X with Python', an answer recommending a package on PyPI is perfectly
valid. That's roughly the role I'd like Pylab to play: you can use
extra packages and still be using Pylab. In this analogy, Scipy is an
important package in the standard library.

Obviously it's not that simple, because of the history behind the name
Scipy, leading to things like the Scipy conferences. But I think what
you're proposing (scipy-the-package within pylab-the-suite within
scipy-the-ecosystem) would make life more confusing, not less. I'm
aiming for scipy-the-package within pylab-the-suite ('Pylab base'?)
within pylab-the ecosystem (aka scipy for historical reasons).

Thanks,
Thomas

[Fernando Perez]

On Sun, Sep 23, 2012 at 2:11 AM, Thomas Kluyver <tak...@gmail.com> wrote:
> I'm
> aiming for scipy-the-package within pylab-the-suite ('Pylab base'?)
> within pylab-the ecosystem (aka scipy for historical reasons).

Yes, that's how I view it too, and I would say Jason's "containment
relation" was inverted. I see the pylab effort as a single
entry-point for users interested in "doing scientific computing with
python", that will:

- point to the various download/distributions that provide the
standard spec (epd, anaconda, sage, pythonxy, C. Gohlke's superpack,
the newly arrived pyzo, etc.)

- provide a basic 'getting started' tutorial and pointers to good
documentation resources, both free and for pay (such as the various
books on the matter that have been published or will be).

- link to scipy-central or some reincarnation thereof as a site for
sharing snippets

- serve as a platform for pep-like discussions to grow standards on
documentation, cross-project help searching, packaging, and whatever
else we feel would improve the experience of users of this "pylab
federation" but that the existing Python standards fail to adequately
meet. Some of these could lead to standards for smooth cross-package
integration akin to what the R community has, and even possibly a
small set of code tools that packages can use to solve these common
problems in a consistent way.

Does that seem sensible?

f

[Thomas Kluyver]

On 23 September 2012 21:49, Fernando Perez <fpere...@gmail.com> wrote:
> Yes, that's how I view it too, and I would say Jason's "containment
> relation" was inverted. I see the pylab effort as a single
> entry-point for users interested in "doing scientific computing with
> python", that will:
>
> - point to the various download/distributions that provide the
> standard spec (epd, anaconda, sage, pythonxy, C. Gohlke's superpack,
> the newly arrived pyzo, etc.)
>
> - provide a basic 'getting started' tutorial and pointers to good
> documentation resources, both free and for pay (such as the various
> books on the matter that have been published or will be).
>
> - link to scipy-central or some reincarnation thereof as a site for
> sharing snippets
>
> - serve as a platform for pep-like discussions to grow standards on
> documentation, cross-project help searching, packaging, and whatever
> else we feel would improve the experience of users of this "pylab
> federation" but that the existing Python standards fail to adequately
> meet. Some of these could lead to standards for smooth cross-package
> integration akin to what the R community has, and even possibly a
> small set of code tools that packages can use to solve these common
> problems in a consistent way.
>
> Does that seem sensible?

Definitely. They are all things I'd like to see happen under the Pylab
umbrella, although #1 is the only one of them I'm actively trying to
achieve at the moment.

Thanks,
Thomas

[Fernando Perez]

On Sun, Sep 23, 2012 at 2:56 PM, Thomas Kluyver <tak...@gmail.com> wrote:
> Definitely. They are all things I'd like to see happen under the Pylab
> umbrella, although #1 is the only one of them I'm actively trying to
> achieve at the moment.

Yup, and that's the right place to start. Every mountain is climbed
one step at a time ;)

[Jason Grout]

Thanks; things are clearer for me now. So you're also proposing that
the scipy conference be renamed the pylab conference?

Thanks,

Jason

[Thomas Kluyver]

On 24 September 2012 12:59, Jason Grout <jason...@creativetrax.com> wrote:
> Thanks; things are clearer for me now. So you're also proposing that the
> scipy conference be renamed the pylab conference?

If the name Pylab gains traction, I hope the conferences (remember
there's EuroScipy and Scipy.in as well as the US one) might be
interested in using it in the future. But there's a lot of history and
reputation around the current name, so any change would probably be a
few years down the line.

I don't think using 'Scipy' for the conferences is a big problem. I
want to present a coherent starting point for newcomers using the name
Pylab, but by the time people are interested enough to go to a
conference, they'll accept the historical reasons behind it being
called Scipy.

Thanks,
Thomas

[Fernando Perez]

On Mon, Sep 24, 2012 at 5:25 AM, Thomas Kluyver <tak...@gmail.com> wrote:
> I don't think using 'Scipy' for the conferences is a big problem. I
> want to present a coherent starting point for newcomers using the name
> Pylab, but by the time people are interested enough to go to a
> conference, they'll accept the historical reasons behind it being
> called Scipy.

I actually think the point of the pylab name is precisely to help the
scipy name continue existing for the library and conferences with less
conflict. Those two uses are well established, and I see no reason to
change them (besides, they fit really well). Just like python.org can
point to the Pycon name for conferences, pylab can point to the scipy
name for the various conferences.

just my take on it

[Thomas Kluyver]

On 25 September 2012 01:10, Fernando Perez <fpere...@gmail.com> wrote:
> I actually think the point of the pylab name is precisely to help the
> scipy name continue existing for the library and conferences with less
> conflict

If we were starting with a blank slate, though, it would make sense to
name the conferences after the stack, not the library.
Scipy-the-package doesn't have an especially prominent role in the
conferences. Of course, the slate is anything but blank.

Thanks,
Thomas

[Fernando Perez]

On Tue, Sep 25, 2012 at 10:19 AM, Thomas Kluyver <tak...@gmail.com> wrote:
> If we were starting with a blank slate, though, it would make sense to
> name the conferences after the stack, not the library.
> Scipy-the-package doesn't have an especially prominent role in the
> conferences. Of course, the slate is anything but blank.

I agree with you there, but given the slate is pretty dirty, I'd say
keeping the name of the conferences as-is, given the reputation
they've built, is not the end of the world.

Cheers,

f

[Min RK]

I think SciPy is a better generic name for everything proposed here. We have historically referred to this as "the SciPy universe" and pylab, to me, still means "Python for MatLab users", a notion at which I bristle.

[Brian Granger]

On Thu, Sep 27, 2012 at 1:50 PM, Min RK <benja...@gmail.com> wrote:
> I think SciPy is a better generic name for everything proposed here. We have historically referred to this as "the SciPy universe" and pylab, to me, still means "Python for MatLab users", a notion at which I bristle.

I agree with this - the "pylab" name does have a bit of a "me too"
matlab copy feel which I don't like.

Cheers,

Brian

> --
>
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgra...@calpoly.edu and elli...@gmail.com

[Aron Ahmadia]

Just a note for those unaware of MATLAB's etymology, it comes from:
MATrix LABoratory

Divesting ourselves from pylab's history as a way to expand the Python
namespace in a MATLAB-like fashion, one could conceivably imagine
pylab to stand for Python Laboratory.

A

On Thu, Sep 27, 2012 at 11:50 PM, Min RK <benja...@gmail.com> wrote:
> I think SciPy is a better generic name for everything proposed here. We have historically referred to this as "the SciPy universe" and pylab, to me, still means "Python for MatLab users", a notion at which I bristle.
>

> --
>
>

[Min RK]

On Thursday, September 27, 2012 1:52:51 PM UTC-7, Aron Ahmadia wrote:

Just a note for those unaware of MATLAB's etymology, it comes from:
MATrix LABoratory

Divesting ourselves from pylab's history as a way to expand the Python
namespace in a MATLAB-like fashion, one could conceivably imagine
pylab to stand for Python Laboratory.

Yes, 'PYthon LABratory' could be a sensible name, were it not that 'pylab' specifically refers to

"This is a matlab(TM) style interface to matplotlib"

(formerly in pylab.py's docstring), and the current prevailing notion that pylab is something that is principally for beginners / MATLAB refugees, and generally avoided after an introductory level.

It seems to me that the case is being made that 'pylab' is a better name for 'the scipy stack', when I think this simply isn't true, nor is increasing the degeneracy of the 'pylab' name particularly helpful.  After this proposal, pylab would refer to at least five things:

1. a MATLAB refugee namespace package, part of matplotlib

2. a mode in IPython for interactive matplotlib, which may or may not involve the above namespace package

3. a website for scientific Python resources

4. a standard set of scientific Python packages, for use in...?

5. a new name for the existing community, already referred to as 'the SciPy community'

I'm not saying that 'SciPy' is the best name for everything, but 'pylab' appears to have exactly every problem 'SciPy' has as a name plus some more, so where is the benefit?

To break it down in terms of Thomas' summary:

To summarise, I view it as three components: the name, the website and 
the standard. 

I am -10 on the name, I think the website is a great idea and extremely important for the future of the community, and I guess I just don't see the value of yet another fairly arbitrary 'standard'.

-MinRK

[Gael Varoquaux]

I actually fully agree with everything Min said. I didn't want to spoil
the party by voicing my opinion too hard, but I think that Min worded it
very well.

My 2 euro cents,

Ga�l

--
Gael Varoquaux
Researcher, INRIA Parietal
Laboratoire de Neuro-Imagerie Assistee par Ordinateur
NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France
Phone: ++ 33-1-69-08-79-68
http://gael-varoquaux.info http://twitter.com/GaelVaroquaux

[James Bergstra]

I can't help but agree with Min as well, fwiw... see a few comments inline:

On Fri, Sep 28, 2012 at 12:50 AM, Gael Varoquaux
<gael.va...@normalesup.org> wrote:
> I actually fully agree with everything Min said. I didn't want to spoil
> the party by voicing my opinion too hard, but I think that Min worded it
> very well.
>
> My 2 euro cents,
>

> Gaël

>
> On Thu, Sep 27, 2012 at 05:49:40PM -0700, Min RK wrote:
>> It seems to me that the case is being made that 'pylab' is a better name for
>> 'the scipy stack', when I think this simply isn't true, nor is increasing the
>> degeneracy of the 'pylab' name particularly helpful. After this proposal,
>> pylab would refer to at least five things:
>
>> 1. a MATLAB refugee namespace package, part of matplotlib

This ship has long sailed.

>> 2. a mode in IPython for interactive matplotlib, which may or may not involve
>> the above namespace package

Also sailed.

>> 3. a website for scientific Python resources

We already have one, not obvious that adding another one will help.

>> 4. a standard set of scientific Python packages, for use in...?

The question mark here seems appropriate. We're 45 messages in on this
thread and it isn't clear what this set is, who would choose it, what
it would apply to (windows? linux distros?). Relative to this,
Anaconda and EPD seem clearly positioned and defined -- what are they
missing?

>> 5. a new name for the existing community, already referred to as 'the SciPy
>> community'

This nails it. I don't get who will benefit from the attempt at
re-branding... maybe the Julia community?

- James

[Eric Firing]

On 2012/09/27 6:50 PM, Gael Varoquaux wrote:
> I actually fully agree with everything Min said. I didn't want to spoil
> the party by voicing my opinion too hard, but I think that Min worded it
> very well.

I agree.

Eric

[Ralf Gommers]

On Fri, Sep 28, 2012 at 7:42 AM, Eric Firing <efirin...@gmail.com> wrote:

On 2012/09/27 6:50 PM, Gael Varoquaux wrote:

I actually fully agree with everything Min said. I didn't want to spoil
the party by voicing my opinion too hard, but I think that Min worded it
very well.

I agree.

Same here. I actually care more that the work on a good website gets done than about the name, but I don't see why that effort couldn't be spent on scipy.org. As for the claim that the SciPy name has been tried and it didn't work, I think scipy.org has simply never been close to the state needed to back up that claim.

Ralf
 

Eric

My 2 euro cents,

Gaël

--

[Dag Sverre Seljebotn]

On 09/27/2012 10:50 PM, Min RK wrote:
> I think SciPy is a better generic name for everything proposed here. We have historically referred to this as "the SciPy universe" and pylab, to me, still means "Python for MatLab users", a notion at which I bristle.
>

I didn't want to spoil the good mood, but now that it's done: Well
said!, and +as-many-as-I-get.

And to deal with SciPy-the-library: Isn't that just an artifact of the
lack of good build&distribution? I mean, if build tools were completely
mature and package distribution very easy (and we somehow fix the
namespacing issue and keep backwards compatability), wouldn't SciPy then
benefit from being broken up into individual "scikits" anyway?

Hopefully, in 2 years the build tool issue is fixed, and then the SciPy
library can disintegrate into its various pieces while the SciPy name is
used for "the coherent whole", including much more than SciPy.

(I really think it would be more likely that I contributed to a
scikit-special (that can still be imported as scipy.special) than SciPy
(e.g., I have wanted to improve the Legendre polynomials, the ones in
there are not accurate for my uses). But the slow release cycles of
SciPy (and the distribution mess in general) means that I can't
immediately benefit from having pushed something "upstream", I still
need to maintain something on the side for my users for some years.)


Dag

[Stéfan van der Walt]

On Fri, Sep 28, 2012 at 1:05 AM, Dag Sverre Seljebotn
<d.s.se...@astro.uio.no> wrote:
> (I really think it would be more likely that I contributed to a
> scikit-special (that can still be imported as scipy.special) than SciPy
> (e.g., I have wanted to improve the Legendre polynomials, the ones in there
> are not accurate for my uses).

Not to hijack the thread, but we're currently working on replacing
these polynomials for use in DiPy, so if you have some good test
cases, please send them my way.

Thanks,
Stéfan

[Joris Van den Bossche]

Op vrijdag 28 september 2012 07:58:49 UTC+2 schreef Ralf Gommers het volgende:

On Fri, Sep 28, 2012 at 7:42 AM, Eric Firing <efirin...@gmail.com> wrote:

On 2012/09/27 6:50 PM, Gael Varoquaux wrote:

I actually fully agree with everything Min said. I didn't want to spoil
the party by voicing my opinion too hard, but I think that Min worded it
very well.

I agree.

Same here. I actually care more that the work on a good website gets done than about the name, but I don't see why that effort couldn't be spent on scipy.org. As for the claim that the SciPy name has been tried and it didn't work, I think scipy.org has simply never been close to the state needed to back up that claim.

Ralf
 

Just speaking as an enthusiastic scientific python user (who follows the lists but never comes to really getting involved).

In the first place, I want to really thank Thomas for starting and getting this discussion forward! I think it is really important to have some more unified starting place for newcomers, with a nice website, a clear set of packages that is adviced to install to start with, good tutorials and examples. And Thomas does a really great job getting this going!

But I also agree with the feeling expressed above about the name 'pylab'. Fortunately, I think the work already done, discussing and defining a standard, thinking about a website, does not depend on the name and is certainly not wasted if this naming discussion starts again (apart from some work for the pylab domain and github account). And I also think Thomas is not 'married' with the name? At a certain point in the discussion, you just proposed to use the name pylab, which I totally understand because otherwise the discussion could go on for ever (but the reservations could maybe been expressed earlier).

What about this? We make a poll (like you did for ipython) with the choices:
- Pylab
- Scipy
- further discuss to look for another name (or otherwise include the other proposals in the poll)
 
As I am just a user, I don't have much credit to speak here, but I wanted to say this.

Kind regards,
Joris Van den Bossche


--
ir. Joris Van den Bossche - PhD Student

KERMIT, Research Unit Knowledge-based Systems
Department of Mathematical Modelling, Statistics and Bioinformatics
Ghent University - Faculty of Bioscience Engineering
Coupure links 653, 9000 Gent, Belgium
URL: http://www.biomath.ugent.be/

[Thomas Kluyver]

On 28 September 2012 01:49, Min RK <benja...@gmail.com> wrote:
> It seems to me that the case is being made that 'pylab' is a better name for
> 'the scipy stack', when I think this simply isn't true, nor is increasing
> the degeneracy of the 'pylab' name particularly helpful. After this
> proposal, pylab would refer to at least five things:

As I've said previously, I think 'Scipy' would be the best name for
this, but we can't effectively develop that as a name for the stack
because of the existence of scipy-the-package. What should a user
expect when they search for 'install scipy', for example? And it would
be highly disruptive to try to rename scipy-the-package.

The existing uses of Pylab are in more or less the same vein as we're
now discussing: a set of key scientific Python packages brought
together in some way. Inevitably there will be some confusion (there
already is, e.g. [1]), but we can make a clear path for something like
'install pylab'. We thought about alternative names, but none of the
suggestions really worked, and many of them conflicted with existing
brands.

The argument about Matlab has already been raised as well. Enough
people feel that Pylab now has some reputation of its own, not just as
'Matlab knock off', that it seemed like the best option.

Unless someone has a fantastic new idea, I'm not really interested in
reopening the naming debate. I think the set of packages has enough
momentum that any reasonable name we use will stick, and we'll never
get anywhere if we keep coming back to this argument.

[1] http://ubuntuforums.org/showthread.php?t=1408261

Thanks,
Thomas

[Brian Granger]

I too agree with Min that the pylab name is not very good.

> As I've said previously, I think 'Scipy' would be the best name for
> this, but we can't effectively develop that as a name for the stack
> because of the existence of scipy-the-package. What should a user
> expect when they search for 'install scipy', for example? And it would
> be highly disruptive to try to rename scipy-the-package.

This is a common issue that companies and open source projects face,
that is, the need to refer to multiple things under one name. The
standard way of handling it is to distinguish the "brand" from the
various "products". Here are some examples:

Brand = Google
Products = Google Docs, Google Plus, Google Search, etc...

Brand = Apple
Products = Apple iPhone, Apple MacBook, Apple iMac

Brand = Apache
Products = HTTP Server, Apache Cassandra, etc.

We simple need to do something like this:

Brand = SciPy
Products = SciPy Library, SciPy Package Set, SciPy Community, SciPy
Conference, etc.

In fact, picking the pylab name (please don't!) doesn't change this at
all. We will face the same exact same need to distinguish the brand
name from the products.

> The existing uses of Pylab are in more or less the same vein as we're
> now discussing: a set of key scientific Python packages brought
> together in some way. Inevitably there will be some confusion (there
> already is, e.g. [1]), but we can make a clear path for something like
> 'install pylab'. We thought about alternative names, but none of the
> suggestions really worked, and many of them conflicted with existing
> brands.
>
> The argument about Matlab has already been raised as well. Enough
> people feel that Pylab now has some reputation of its own, not just as
> 'Matlab knock off', that it seemed like the best option.
>
> Unless someone has a fantastic new idea, I'm not really interested in
> reopening the naming debate. I think the set of packages has enough
> momentum that any reasonable name we use will stick, and we'll never
> get anywhere if we keep coming back to this argument.

Personally, I think we don't need a new name, SciPy is well
established and a decent name all things considered.

Cheers,

Brian

> [1] http://ubuntuforums.org/showthread.php?t=1408261
>
> Thanks,
> Thomas
>

[Denis Laxalde]

Hi,

Thomas Kluyver wrote:
> As I've said previously, I think 'Scipy' would be the best name for
> this, but we can't effectively develop that as a name for the stack
> because of the existence of scipy-the-package. What should a user
> expect when they search for 'install scipy', for example?

Actually looking for 'install scipy' in google basically leads to
<http://www.scipy.org/Installing_SciPy>, which, for most OS gives
detailed instructions to install both numpy and scipy (the library).
So, it seems already pretty close to the scipy environment that a
newcomer might be looking for, doesn't it?

> Unless someone has a fantastic new idea, I'm not really interested in
> reopening the naming debate. I think the set of packages has enough
> momentum that any reasonable name we use will stick, and we'll never
> get anywhere if we keep coming back to this argument.

Not sure it's fantastic, but what about a self-explanatory name like
"scipy-platform" for the metapackage?

Then, concerning the scipy.org website, a short introduction stating
first that � SciPy is a platform for scientific computing in Python �
and then giving a few details about its components (pretty much like
new.scipy.org does now) should be sufficiently clear for new potential
user. In the download section, you'd see something like "Download the
SciPy Platform: a complete environment for scientific computing in
Python (or SciPy: batteries included)".

Just my 2c...

--
Denis Laxalde

[eat]

Hi,

On Fri, Sep 28, 2012 at 6:09 PM, Brian Granger <elli...@gmail.com> wrote:

I too agree with Min that the pylab name is not very good.

> As I've said previously, I think 'Scipy' would be the best name for
> this, but we can't effectively develop that as a name for the stack
> because of the existence of scipy-the-package. What should a user
> expect when they search for 'install scipy', for example? And it would
> be highly disruptive to try to rename scipy-the-package.

This is a common issue that companies and open source projects face,
that is, the need to refer to multiple things under one name.  The
standard way of handling it is to distinguish the "brand" from the
various "products".  Here are some examples:

Brand = Google
Products = Google Docs, Google Plus, Google Search, etc...

Brand = Apple
Products = Apple iPhone, Apple MacBook, Apple iMac

Brand = Apache
Products = HTTP Server, Apache Cassandra, etc.

We simple need to do something like this:

Brand = SciPy
Products = SciPy Library, SciPy Package Set, SciPy Community, SciPy
Conference, etc.

For a simple user makes a lot sense!

+ FWIWO,

-eat

--

[Charles R Harris]

OT: To continue the hijack, where is that work taking place? The mailing list doesn't show any recent mentions.

Chuck

[Eric Firing]

On 2012/09/28 12:32 AM, Thomas Kluyver wrote:
> Unless someone has a fantastic new idea, I'm not really interested in
> reopening the naming debate. I think the set of packages has enough
> momentum that any reasonable name we use will stick, and we'll never
> get anywhere if we keep coming back to this argument.

There is a bit more than that to the recent set of posts on this thread.

I think everyone has agreed strongly that a greatly improved web site is
needed.

The disagreement about name is in part a question of strategy: evolution
(drastically improve the scipy.org site) or revolution (start from
scratch with a brand-new pylab.org site).

There is also substantial disagreement about the advisability of trying
to establish "the standard" as a part of this effort. If it goes
forward, the question of what goes into "the standard" will likely
stimulate heated controversy. This is potentially a huge can of worms
that could spoil the web site dinner.

All of us appreciate your initiative, and really don't want us to get
bogged down in quibbling over a name; but I think those of us who agree
with Min are concerned that your initiative and energy might be wasted
if you (and we) try to do too much at once, without a sufficient level
of agreement. I am not saying you have to wait for "consensus"--but I
think the present level of disagreement is a danger sign.

Eric

[Fernando Perez]

On Fri, Sep 28, 2012 at 12:06 PM, Eric Firing <efirin...@gmail.com> wrote:
> All of us appreciate your initiative, and really don't want us to get bogged
> down in quibbling over a name; but I think those of us who agree with Min
> are concerned that your initiative and energy might be wasted if you (and
> we) try to do too much at once, without a sufficient level of agreement. I
> am not saying you have to wait for "consensus"--but I think the present
> level of disagreement is a danger sign.

FWIW, add my +1 to Min's reboot: last time I participated I'd decided
to go along with the pylab name mostly to avoid what could seem to be
a circular discussion esp. when I wasn't sure if I was the only one
feeling that way. I also wanted to give Thomas as much leeway as
possible, following the idea that the person leading the hard work
should have some freedom to also make some final calls. But just the
day before I'd been chatting with Stefan and Jarrod precisely about
how ultimately I thought we'd be better off sticking with 'scipy' and
dealing with the not-perfect-but-manageable situation regarding the
package vs community questions.

Thomas, indeed I think none of us wants to turn this into endless
circular discussion, and we're *enormously* grateful that you're
tackling this critically important issue. Please don't mistake this
as any attempt by anyone to torpedo this great effort you're leading.
But as Eric wisely indicates, perhaps this much dissent should be
taken into consideration.

Cheers,

f

[Brian Granger]

What about putting together an online survey and sending it to the
various mailing lists to try and get a better read on the communities
thoughts on all of this. We could ask about the name issue, as well
as things related to the website and packaging.

[Benjamin Root]

Just a quick re-push for "unipy"?

Anyway, the survey is a good idea.  I am also going to try and poll coworkers who merely use the stack and may not know the history for their perspectives.

Ben Root

[Anthony Scopatz]

Only if our mascot can be Captain One-Py, the Pyrate scourge of the Seven Cs. 

 

Anyway, the survey is a good idea.  I am also going to try and poll coworkers who merely use the stack and may not know the history for their perspectives.

+1 to a survey, though I am unsure of its ultimate impact.

Be Well

Anthony 

 

Ben Root

--
 
 

[Benjamin Root]

No objections here.  Can he have a python on his shoulder?

 

Anyway, the survey is a good idea.  I am also going to try and poll coworkers who merely use the stack and may not know the history for their perspectives.

+1 to a survey, though I am unsure of its ultimate impact.

Well, scurvy tends to lead to sickness and ultimately.... Oh, misread that. Carry on!

Ben Root

[Thomas Kluyver]

On Friday, September 28, 2012 8:17:32 PM UTC+1, Fernando Perez wrote:

Thomas, indeed I think none of us wants to turn this into endless
circular discussion, and we're *enormously* grateful that you're
tackling this critically important issue.  Please don't mistake this
as any attempt by anyone to torpedo this great effort you're leading.
But as Eric wisely indicates, perhaps this much dissent should be
taken into consideration.

If there's this much depth of feeling, I'll reconsider. But if we're going for Scipy, we need some kind of confirmation from the authors of Scipy-the-package that they don't mind ceding the primary use of the name, i.e. when people look for 'installing Scipy', the primary resource would be about installing the complete stack, not the package. And I think we need to do this now, not in a couple more years when the packaging mess might just have improved.

I appreciate the suggestion from Brian of naming things 'scipy library', 'scipy stack', etc., but I don't think we have the leverage to put that distinction in people's consciousness - they'll just call something 'scipy', without being quite clear on what they mean.

Eric: the question of the standard has *already* stimulated lots of debate. We mostly seem to have an agreement on a set of packages based on a slide Fernando provided. I think the standard is a very important part of this: it establishes a base so people can say 'it runs on Scipy/Pylab 2.0', without having to either list dependencies or suggest others get the same distribution they use.

Denis:

> Actually looking for 'install scipy' in google basically leads to
> <http://www.scipy.org/Installing_SciPy>, which, for most OS gives
> detailed instructions to install both numpy and scipy (the library).
> So, it seems already pretty close to the scipy environment that a
> newcomer might be looking for, doesn't it?

* Newcomers are unlikely to want to compile from source. The download page is a better starting point, but:
* It assumes the user already has Python (far from a given for Windows users)
* It doesn't mention all the important packages beyond numpy+scipy. We want the base to have a much more complete set of tools.
* It only very briefly mentions Python distributions, which are probably the easiest way to get the complete stack for most newcomers.
* Linux packages, the easiest way in for users on Linux, are only described under 'unofficial releases', and link to worryingly out of date instructions (about Ubuntu 6.06, for instance).
* The 'official' download links point to the ugly, unfriendly Sourceforge files pages, where the user has to drill down two folder levels and then pick the correct file by filename.
* ...and the scipy.org server frequently goes down.

Thomas

[Ralf Gommers]

On Sat, Sep 29, 2012 at 12:06 PM, Thomas Kluyver <tak...@gmail.com> wrote:

On Friday, September 28, 2012 8:17:32 PM UTC+1, Fernando Perez wrote:

Thomas, indeed I think none of us wants to turn this into endless
circular discussion, and we're *enormously* grateful that you're
tackling this critically important issue.  Please don't mistake this
as any attempt by anyone to torpedo this great effort you're leading.
But as Eric wisely indicates, perhaps this much dissent should be
taken into consideration.

If there's this much depth of feeling, I'll reconsider. But if we're going for Scipy, we need some kind of confirmation from the authors of Scipy-the-package that they don't mind ceding the primary use of the name, i.e. when people look for 'installing Scipy', the primary resource would be about installing the complete stack, not the package.

Sounds great to me. If both scipy.org and googling "installing scipy" leads people to EPD/Pythonxy/... it would solve a lot of headaches with installing scipy-the-package.

Ralf

 

And I think we need to do this now, not in a couple more years when the packaging mess might just have improved.

I appreciate the suggestion from Brian of naming things 'scipy library', 'scipy stack', etc., but I don't think we have the leverage to put that distinction in people's consciousness - they'll just call something 'scipy', without being quite clear on what they mean.

Eric: the question of the standard has *already* stimulated lots of debate. We mostly seem to have an agreement on a set of packages based on a slide Fernando provided. I think the standard is a very important part of this: it establishes a base so people can say 'it runs on Scipy/Pylab 2.0', without having to either list dependencies or suggest others get the same distribution they use.

Denis:

> Actually looking for 'install scipy' in google basically leads to
> <http://www.scipy.org/Installing_SciPy>, which, for most OS gives
> detailed instructions to install both numpy and scipy (the library).
> So, it seems already pretty close to the scipy environment that a
> newcomer might be looking for, doesn't it?

* Newcomers are unlikely to want to compile from source. The download page is a better starting point, but:
* It assumes the user already has Python (far from a given for Windows users)
* It doesn't mention all the important packages beyond numpy+scipy. We want the base to have a much more complete set of tools.
* It only very briefly mentions Python distributions, which are probably the easiest way to get the complete stack for most newcomers.
* Linux packages, the easiest way in for users on Linux, are only described under 'unofficial releases', and link to worryingly out of date instructions (about Ubuntu 6.06, for instance).
* The 'official' download links point to the ugly, unfriendly Sourceforge files pages, where the user has to drill down two folder levels and then pick the correct file by filename.
* ...and the scipy.org server frequently goes down.

Thomas

--
 
 

[Chris Kees]

Thomas,

Could you please post the current description of the stack that you think is gathering some momentum or remind  me where it is?

Thanks,

Chris

[Thomas Kluyver]

Hi Chris,

On Saturday, September 29, 2012 4:38:00 PM UTC+1, Chris Kees wrote:

Could you please post the current description of the stack that you think is gathering some momentum or remind  me where it is?

The list of packages here, plus h5py and nose, and after a lot of discussion, it seems that we'd want to specify the dependencies for IPython notebook (pyzmq and tornado):
https://github.com/pylab/website/blob/master/specification.rst

On a separate matter, there's been a possible replacement for the Scipy website lurking at new.scipy.org or scipy.github.com for as long as I've been interested in it. Does anyone know what the status of that is? Is it still intended to replace the current scipy.org? If so, what still needs to be done?

Thanks,
Thomas

[Ralf Gommers]

It is intended to replace scipy.org. See http://article.gmane.org/gmane.comp.python.scientific.devel/16884.

Ralf
 

[Ralf Gommers]

On Fri, Sep 28, 2012 at 10:05 AM, Dag Sverre Seljebotn <d.s.se...@astro.uio.no> wrote:

On 09/27/2012 10:50 PM, Min RK wrote:

I think SciPy is a better generic name for everything proposed here.  We have historically referred to this as "the SciPy universe" and pylab, to me, still means "Python for MatLab users", a notion at which I bristle.

I didn't want to spoil the good mood, but now that it's done: Well said!, and +as-many-as-I-get.

And to deal with SciPy-the-library: Isn't that just an artifact of the lack of good build&distribution? I mean, if build tools were completely mature and package distribution very easy (and we somehow fix the namespacing issue and keep backwards compatability), wouldn't SciPy then benefit from being broken up into individual "scikits" anyway?

Probably, to some extent. The scipy submodules aren't all independent though, so the gain may be limited in practice. Example: scipy.stats depends on interpolate, integrate, linalg, optimize, misc and special.

Hopefully, in 2 years the build tool issue is fixed, and then the SciPy library can disintegrate into its various pieces while the SciPy name is used for "the coherent whole", including much more than SciPy.

(I really think it would be more likely that I contributed to a scikit-special (that can still be imported as scipy.special) than SciPy (e.g., I have wanted to improve the Legendre polynomials, the ones in there are not accurate for my uses). But the slow release cycles of SciPy (and the distribution mess in general) means that I can't immediately benefit from having pushed something "upstream", I still need to maintain something on the side for my users for some years.)

Some years? The goal for SciPy is a release every 6 months. In the last 2.5 years we've had 0.7.2, 0.8.0, 0.9.0, 0.10.0, 0.10.1 and 0.11.0, so we're not too far off that goal. By breaking up SciPy you could see somewhat faster releases of some of the components, but certainly not more than twice faster without giving up on the current level of QA.

Contributing to something as widely used as scipy.special is inherently more work than just doing something for yourself. That's not going to change by breaking up Scipy. The main advantage of a possible break-up I see is that you can have a more specialized mailing list and website - that seems to help for scikits-image for example.

Ralf

[Gael Varoquaux]

Hi,

Just to give a bit more information on my 'vote' against the 'pylab'
name, my reasons to dislike it is two fold:

1. I am weary of the name collision. In my experience name collisions are
very costly. Google, books, articles, teaching material, people's
memory, ... add inertia and confusion.

2. There has already been a lot of investment in the "scipy" name
(website, conference), so if we are going to use something that has a
name collision, we might as well use it.

My favorite name option would be 'scipylab', but I have always had the
impression that people saw it as too much of a 'designed-by-committee'
name.

If we go with 'scipy', I don't think that the collision matters that
much. Most often, people who think that they want to install 'scipy'
really should be installing a larger stack. And if they are routed
towards installing EPD, Anaconda, Python(x,y) or WinPython, that's
probably a good thing.

Anyhow, I don't want to weight in too much on the name discussion,
because all in all what matters most is brilliant people with energy and
enthusiasm to get things going. So, I'll bow to the rules of a do-cracy:
those who do the work get to decide. Thanks to Thomas and Almar, and all
the lot pushing things forward! Having good community on a united
ecosystem (or 'stack') is very important.

Thanks for having an open and enthusiastic discussion.

Ga�l

[Travis Oliphant]

I realized yesterday that I wasn't getting emails from this discussion, so I missed the arguments against pylab.

I am generally pretty agnostic when it comes to names, and I agree we have all invested a lot in the SciPy brand. It is however important to name things in distinctive ways.

I am hopeful that we can continue to use SciPy to describe the conference and the package.

However we still need another name to describe a standard collection of packages which is where this conversation started and what I still hope we resolve

apt-get install scipy
yum install scipy

already mean something and it isn't install a large collection of other packages. It might be nice if scipy were already a collection of packages --- but it isn't and we need a name to describe the meta-package.

Using the name pylab for that collection makes a lot of sense as it also has name recognition and is already commonly used to connote an environment utilizing several packages -- the concern that it sounds like Matlab is not persuasive to me.

But as I said, we could name that meta package something else as well, other reasonable suggestions:

unipy
scipylab --- not bad actually
scipyfull
scipystack
scipysys
...

So, those who are opposed to the pylab name still need to suggest another name. I don't see how it will work to accomplish the intent of this effort without another name.

Thanks,

Travis





--
Travis Oliphant
(on a mobile)
512-826-7480

> Gaël

[Ralf Gommers]

On Sun, Sep 30, 2012 at 5:43 PM, Travis Oliphant <tra...@continuum.io> wrote:

I realized yesterday that I wasn't getting emails from this discussion, so I missed the arguments against pylab.

Archive of this discussion: https://groups.google.com/forum/?fromgroups#!topic/numfocus/aQKHmlS4m0Y

 

I am generally pretty agnostic when it comes to names, and I agree we have all invested a lot in the SciPy brand.  It is however important to name things in distinctive ways.

I am hopeful that we can continue to use SciPy to describe the conference and the package.

However we still need another name to describe a standard collection of packages which is where this conversation started and what I still hope we resolve

apt-get install scipy
yum install scipy

already mean something and it isn't install a large collection of other packages.   It might be nice if scipy were already a collection of packages --- but it isn't and we need a name to describe the meta-package.

Using the name pylab for that collection makes a lot of sense as it also has name recognition and is already commonly used to connote an environment utilizing several packages -- the concern that it sounds like Matlab is not persuasive to me.

But as I said, we could name that meta package something else as well,  other reasonable suggestions:

 unipy
 scipylab --- not bad actually
 scipyfull
 scipystack
 scipysys
 ...

So, those who are opposed to the pylab name still need to suggest another name.  I don't see how it will work to accomplish the intent of this effort without another name.

Brian Granger summarized it pretty well: we have a SciPy Community, SciPy Conferences, SciPy Library and SciPy Package. I'd call the larger collection of packages "SciPy Stack" probably, but "SciPy Package" works fine too.

Ralf

 

[R. Michael Weylandt]

On Sun, Sep 30, 2012 at 5:07 PM, Ralf Gommers <ralf.g...@gmail.com> wrote:
>
>
> On Sun, Sep 30, 2012 at 5:43 PM, Travis Oliphant <tra...@continuum.io>

>> I am hopeful that we can continue to use SciPy to describe the conference
>> and the package.
>>
>> However we still need another name to describe a standard collection of
>> packages which is where this conversation started and what I still hope we
>> resolve
>>
>> apt-get install scipy
>> yum install scipy
>>
>> already mean something and it isn't install a large collection of other
>> packages. It might be nice if scipy were already a collection of packages
>> --- but it isn't and we need a name to describe the meta-package.
>>
>> Using the name pylab for that collection makes a lot of sense as it also
>> has name recognition and is already commonly used to connote an environment
>> utilizing several packages -- the concern that it sounds like Matlab is not
>> persuasive to me.
>>
>> But as I said, we could name that meta package something else as well,
>> other reasonable suggestions:
>>
>> unipy
>> scipylab --- not bad actually
>> scipyfull
>> scipystack
>> scipysys
>> ...
>>
>

> Brian Granger summarized it pretty well: we have a SciPy Community, SciPy
> Conferences, SciPy Library and SciPy Package. I'd call the larger collection
> of packages "SciPy Stack" probably, but "SciPy Package" works fine too.
>

Silent user chiming in to say I quite like the sound of something like

apt-get install scipy-stack

and feel that one man's "name collision" is another's synergy. It also
lets EPD et al say things like "includes the full SciPy stack," which
does't sound too bad and is pretty clear that there's more than just
scipy in it.

Now I'll cede the floor back to the big dogs,

Michael

[Travis Oliphant]

>
> Silent user chiming in to say I quite like the sound of something like
>
> apt-get install scipy-stack
>
> and feel that one man's "name collision" is another's synergy. It also
> lets EPD et al say things like "includes the full SciPy stack," which
> does't sound too bad and is pretty clear that there's more than just
> scipy in it.

This sounds fine to me. I think either scipy-stack or scipystack would work. I'm interested in what others think as well. Thomas, do you have another name you prefer?

-Travis

[Almar Klein]

I did read the arguments against the name pylab. I had a gut feeling against the name scipy, but I could not put my finger on it. A few posts further  Thomas and Travis clarified my concerns a bit.

But first a little rant of how I view this thing that we try to achieve:

Over the years, the scientific Python ecosystem and community have grown a lot. When I entered this world 4 years ago, it was good, but you needed to be a bit of a geek to find your way around. Many users were also developers, often out of necessity. A lot of work has been done since then, and we're becoming more 'mature'. Now is the time that we say, hey you other scientists, we've got something great going on, come join us, programming is fun again! To me, this is what the pylab/scipy idea is about.

My concern with the name scipy comes down to the fact that we want to do something new and fresh, and I'm afraid that using the existing scipy name inhibits us from really doing what we want to do. Having said that, I'm also compelled by Brians argument of scipy-stack, scipy-library, scipy-conference, scipy-community.

The first thing is the website. Who runs that now? If we're going to use scipy as the name, I think Thomas et al should have the freedom to start fresh. Not improve the current one, but to start with a clean sheet. Is that possible?

Further ...

Travis said:

However we still need another name to describe a standard collection of packages which is where this conversation started and what I still hope we resolve.

 

Not necessarily: we could have scipy as name for the umbrella idea,  scipy-stack for what we want to do here, and scipy-lib (or similar) for what is now the scipy package.

Thomas wrote:

As I've said previously, I think 'Scipy' would be the best name for
this, but we can't effectively develop that as a name for the stack
because of the existence of scipy-the-package. What should a user
expect when they search for 'install scipy', for example? And it would
be highly disruptive to try to rename scipy-the-package.

 

How disruptive would it be? Would it be worth it? Either we use scipy and have to change the meaning and names of some of the things that we now call scipy (mainly the package). Or we use pylab, and don't have these disruptions, but we have a new name. I personally don't think a new name is a bad thing (and I dont think it makes us look like Matlab wannabees :))

  Almar

[Eric Firing]

On 2012/09/29 11:44 PM, Thomas Kluyver wrote:
> Hi Chris,
>
> On Saturday, September 29, 2012 4:38:00 PM UTC+1, Chris Kees wrote:
>
> Could you please post the current description of the stack that you
> think is gathering some momentum or remind me where it is?
>
>
> The list of packages here, plus h5py and nose, and after a lot of
> discussion, it seems that we'd want to specify the dependencies for
> IPython notebook (pyzmq and tornado):
> https://github.com/pylab/website/blob/master/specification.rst

Thomas,

Thank you. I think it would be very helpful to include some rationale
for deciding what is included and what is not. As it stands, I have no
idea what that rationale is. The first five items--down through
ipython--are obvious. They form a fairly general base, rather like a
basic Matlab installation. The remainder look like a subset of possible
choices of more specialized capabilities. Why this particular subset?
Are you sure it is wise to try, at this stage, to lock together a core
with some such subset?

I seem to have missed the "lot of discussion" of package selection. Did
it occur on the scipy list?

Eric

[Dag Sverre Seljebotn]

I apologise for making such a baseless comment.

I didn't quite mean (I hope) "two years between SciPy releases", I think
I meant to allude to the problem that users don't upgrade often enough,
the distribution mess etc. I guess the idea was I could more easily get
people to upgrade their "skspecial" to something I just pushed, than
their "SciPy". But I see the error in that kind of reasoning now.

Dag Sverre

[Ralf Gommers]

On Sun, Sep 30, 2012 at 8:25 PM, Almar Klein <almar...@gmail.com> wrote:

I did read the arguments against the name pylab. I had a gut feeling against the name scipy, but I could not put my finger on it. A few posts further  Thomas and Travis clarified my concerns a bit.

But first a little rant of how I view this thing that we try to achieve:

Over the years, the scientific Python ecosystem and community have grown a lot. When I entered this world 4 years ago, it was good, but you needed to be a bit of a geek to find your way around. Many users were also developers, often out of necessity. A lot of work has been done since then, and we're becoming more 'mature'. Now is the time that we say, hey you other scientists, we've got something great going on, come join us, programming is fun again! To me, this is what the pylab/scipy idea is about.

My concern with the name scipy comes down to the fact that we want to do something new and fresh, and I'm afraid that using the existing scipy name inhibits us from really doing what we want to do. Having said that, I'm also compelled by Brians argument of scipy-stack, scipy-library, scipy-conference, scipy-community.

The first thing is the website. Who runs that now? If we're going to use scipy as the name, I think Thomas et al should have the freedom to start fresh. Not improve the current one, but to start with a clean sheet. Is that possible?

scipy.org is a wiki, the plan was/is to flip the switch and make it point to http://scipy.github.com/. That is maintained by the "scipy web team". Everyone can make pull requests against https://github.com/scipy/scipy.org-new to update scipy.github.com, and people interested to do more can join the web team to get commit rights.

Starting from scratch instead of improving what's there is also a good option imho. If it makes scipy.org a better portal to our community, I don't think anyone will object.

Ralf

Further ...

Travis said:

However we still need another name to describe a standard collection of packages which is where this conversation started and what I still hope we resolve.

 

Not necessarily: we could have scipy as name for the umbrella idea,  scipy-stack for what we want to do here, and scipy-lib (or similar) for what is now the scipy package.

Thomas wrote:

As I've said previously, I think 'Scipy' would be the best name for
this, but we can't effectively develop that as a name for the stack
because of the existence of scipy-the-package. What should a user
expect when they search for 'install scipy', for example? And it would
be highly disruptive to try to rename scipy-the-package.

 

How disruptive would it be? Would it be worth it?

Very, and no (imho).
 

Either we use scipy and have to change the meaning and names of some of the things that we now call scipy (mainly the package).

We actually don't. The name SciPy has been used for the community, conferences, scientific python stack as well as the package for a long time. We just never had a good website or a meta-package.

Ralf


 

Or we use pylab, and don't have these disruptions, but we have a new name. I personally don't think a new name is a bad thing (and I dont think it makes us look like Matlab wannabees :))

  Almar

--
 
 

[Thomas Kluyver]

On Sunday, 30 September 2012 19:24:56 UTC+1, Travis wrote:

Thomas, do you have another name you prefer?

In terms of completely new names, I quite liked pyscis (which I'd pronounce py-skees), but it's too close to the existing package PySCeS. I think more formally defining what we already call the 'scipy stack' is the best option.

As for metapackage naming, scipy-stack could work. A rough precedent would be the r-recommended package in Ubuntu.

Eric:

> I seem to have missed the "lot of discussion" of package selection.  Did it occur on the scipy list?

Yes, scipy-user. If you want to read through all the discussion, it starts here: http://article.gmane.org/gmane.comp.python.scientific.user/32620

Much of the credit for reaching that consensus is Fernando's - the list of packages is based on a slide he showed. There was discussion of having multiple 'levels' of the standard, e.g. 'base' vs. 'recommended'. One poster suggested no less than four levels, which I vetoed. Fernando brought some light to this again, by suggesting that we define a set without a compiler, and a larger set which relies on having a compiler installed. This had been another point of contention: tools like Cython are very valuable, but requiring compliant distributions to integrate a compiler would make it much harder to meet the standard.

To be clear, the list of packages I mentioned above is the 'without compiler' set - I focused on defining this first, because of doing one step at a time.

Almar:
Thanks for the vote of confidence, but I'd rather not create another new scipy.org site. The existing 'new' site (http://scipy.github.com/ ) should be a substantial improvement, so I'd like to focus on getting that in place at scipy.org and improving it. I'm not really a web designer (although I'm pleased to see several other projects borrowing the design from ipython.org, which I was involved with).

Specifically, here's what I think we need to do:
- Get the 'new' scipy site up at scipy.org
- Turn the 'download' page into an 'install' page, which has instructions/links for different platforms, starting with the most convenient ways - Python distributions and Linux distro packages, not source tarballs and VCS checkouts.
- Finish defining a standardised 'scipy stack', and promote this concept. We're only going to spread by word of mouth, so we need words for people to say.
- Build tutorials around the new scipy stack.

Thanks,
Thomas

[Fernando Perez]

On Sun, Sep 30, 2012 at 11:54 AM, Eric Firing <efirin...@gmail.com> wrote:
> Thank you. I think it would be very helpful to include some rationale for
> deciding what is included and what is not. As it stands, I have no idea
> what that rationale is. The first five items--down through ipython--are
> obvious. They form a fairly general base, rather like a basic Matlab
> installation. The remainder look like a subset of possible choices of more
> specialized capabilities. Why this particular subset? Are you sure it is
> wise to try, at this stage, to lock together a core with some such subset?

I just wanted to add a minor clarification from when I wrote that
slide that Thomas mentioned
(https://speakerdeck.com/u/fperez/p/1204-biofrontiers-boulder?slide=21).
I had drawn a line between the 'core' packages that was basically EPD
free + sympy, and the layer below is what I consider a reasonable
'full stack' that is widely applicable, stands well against Matlab and
R even with add-ons, and yet is not too discipline-specific.

My thought when I wrote that slide was that by adding sympy to the
contents of EPD free, it gave a really solid and still very
lightweight suite that was very useful for a lot of work at least up
to the undergraduate college level and likely a lot of industry.
Given that Matlab's symbolic computing is a kludgy add-on and that R
also doesn't have a really native solution, this seemed like a good
target.

I figured that it might be possible to engage Enthought in the
conversation to include Sympy in EPD free, as it would round up a
really solid core without stretching the boundaries of EPD free too
far beyond where they are today.

I also hope that we'll engage better Enthought and the other players
who have distribution projects; as Nathaniel wisely indicated in a
prior email, it's important that we work *with* all the participants
who have made already lots of progress on these problems and not that
we appear to be handing anyone a 'this is what you must do' task list.
Nathaniel made those points eloquently and I won't repeat it here,
just wanted to say I think he nailed a key point in this effort.

Cheers,

f

[Travis Oliphant]

I like the notion of a small stack that includes SymPy.

For the larger stack, MayaVi which you include on your slide (while a very nice tool) is pretty heavy as a dependency --- it requires VTK and a GUI.

I would hesitate putting it into the automatically included list.

--
Travis Oliphant
(on a mobile)
512-826-7480

> --
>
>

[Fernando Perez]

On Sun, Sep 30, 2012 at 6:11 PM, Travis Oliphant <tra...@continuum.io> wrote:
> I like the notion of a small stack that includes SymPy.
>
> For the larger stack, MayaVi which you include on your slide (while a very nice tool) is pretty heavy as a dependency --- it requires VTK and a GUI.
>
> I would hesitate putting it into the automatically included list.

No worries: as part of the longer thread you missed, we'd pretty much
agreed that despite its value, the VTK/GUI combination made mayavi too
tall an order for the spec. Great to have and something distributions
can tackle if they want, but too much to bite for now.

Cheers,

f

[Almar Klein]

Fernando: 

I had drawn a line between the 'core' packages that was basically EPD
free + sympy, and the layer below is what I consider a reasonable
'full stack' that is widely applicable, stands well against Matlab and
R even with add-ons, and yet is not too discipline-specific.

I think that defining a relatively basic core stack is a good idea. However, I have my doubts about defining a 'full stack'. The reason is that it would be very difficult to define one that is both complete and not too discipline-specific. It will either be incomplete or too large. Instead I propose to define multiple categories for the different disciplines. What packages go in these categories can be specified by people from these fields themselves. I've heard the R uses something like this as well.

I believe that such an approach also scales better. As Python is adopted in new fields of science, we don't have to extend the full-stack, but can simply define a new category.

Of course, it's difficult to draw a line between categories, but that's ok. categories can overlap; one package can be a part of different categories. Although we probably need to find a way to ensure that two categories having the same package use the same version of that package.

But let's first reach an agreement on the core stack...

 

Thomas wrote:

As I've said previously, I think 'Scipy' would be the best name for
this, but we can't effectively develop that as a name for the stack
because of the existence of scipy-the-package. What should a user
expect when they search for 'install scipy', for example? And it would
be highly disruptive to try to rename scipy-the-package.

 

How disruptive would it be? Would it be worth it?

Very, and no (imho).

So "import scipy" will always import scipy-the-package. I suppose that is acceptable, since "import scipystack" does not make a lot of sense. Nevertheless, I think that to avoid confusion we should change how we refer to scipy-the-package --- we're already doing that now, but I don't like 'scipy-the-package'. I'd say we call it scipy-package or scipy-lib.

Thomas: 

Specifically, here's what I think we need to do:
- Get the 'new' scipy site up at scipy.org
- Turn the 'download' page into an 'install' page, which has instructions/links for different platforms, starting with the most convenient ways - Python distributions and Linux distro packages, not source tarballs and VCS checkouts.
- Finish defining a standardised 'scipy stack', and promote this concept. We're only going to spread by word of mouth, so we need words for people to say.
- Build tutorials around the new scipy stack.

From this I understand that you're going with the scipy name? If so, I agree; let's move on to the next topic :)

  Almar

[Robert Bradshaw]

On Fri, Sep 28, 2012 at 8:09 AM, Brian Granger <elli...@gmail.com> wrote:
> I too agree with Min that the pylab name is not very good.

>
>> As I've said previously, I think 'Scipy' would be the best name for
>> this, but we can't effectively develop that as a name for the stack
>> because of the existence of scipy-the-package. What should a user
>> expect when they search for 'install scipy', for example? And it would
>> be highly disruptive to try to rename scipy-the-package.
>

> This is a common issue that companies and open source projects face,
> that is, the need to refer to multiple things under one name. The
> standard way of handling it is to distinguish the "brand" from the
> various "products". Here are some examples:
>
> Brand = Google
> Products = Google Docs, Google Plus, Google Search, etc...
>
> Brand = Apple
> Products = Apple iPhone, Apple MacBook, Apple iMac
>
> Brand = Apache
> Products = HTTP Server, Apache Cassandra, etc.
>
> We simple need to do something like this:
>
> Brand = SciPy
> Products = SciPy Library, SciPy Package Set, SciPy Community, SciPy
> Conference, etc.
>
> In fact, picking the pylab name (please don't!) doesn't change this at
> all. We will face the same exact same need to distinguish the brand
> name from the products.
>
>> The existing uses of Pylab are in more or less the same vein as we're
>> now discussing: a set of key scientific Python packages brought
>> together in some way. Inevitably there will be some confusion (there
>> already is, e.g. [1]), but we can make a clear path for something like
>> 'install pylab'. We thought about alternative names, but none of the
>> suggestions really worked, and many of them conflicted with existing
>> brands.
>>
>> The argument about Matlab has already been raised as well. Enough
>> people feel that Pylab now has some reputation of its own, not just as
>> 'Matlab knock off', that it seemed like the best option.
>>
>> Unless someone has a fantastic new idea, I'm not really interested in
>> reopening the naming debate. I think the set of packages has enough
>> momentum that any reasonable name we use will stick, and we'll never
>> get anywhere if we keep coming back to this argument.
>
> Personally, I think we don't need a new name, SciPy is well
> established and a decent name all things considered.

+1

I have had reservations about the new name as well, but haven't spoken
up 'till now. Branding things like this is IMHO a fine solution to the
potential confusion between scipy-the-library and scipy-the-stack.

- Robert

[Thomas Kluyver]

On Monday, 1 October 2012 09:13:00 UTC+1, Almar Klein wrote:

I think that defining a relatively basic core stack is a good idea. However, I have my doubts about defining a 'full stack'. The reason is that it would be very difficult to define one that is both complete and not too discipline-specific. It will either be incomplete or too large. Instead I propose to define multiple categories for the different disciplines. What packages go in these categories can be specified by people from these fields themselves. I've heard the R uses something like this as well.

To be clear, I don't think we ever want to define a 'full' stack, in the sense of 'this is all you need'. But at the same time, I'd hate for a newcomer's first experience with the stack to be wading into the Python Packaging Mess. The base should allow people to do useful work, rather than being a 'core' in the sense of the bare minimum. It should also allow you to share code that 'runs on the Scipy stack'.

To my mind, the DataFrame stuff from Pandas is a key part of that - column based data in CSV files is fundamental to a lot of things, and it's important that we have a good way to handle it. Statsmodels is a natural compliment to that, although I've yet to work out how to do what I want with it. I'm trying to work out how widely used the other packages are, but I'm hampered by my limited view of science - I don't handle anything remotely approaching 'big data', for example. Should we consider HDF5 files fundamental? Image processing? Machine learning? Graphs (in the NetworkX sense)?

I'd welcome input on any of this - there wasn't much discussion of some of these packages on scipy-user. But I'd advise against returning to a minimalist 'core' standard that everyone will want to add bits to. To summarise how I see the input so far:
- "Core packages": Python, Numpy, Scipy, Matplotlib, IPython
- Fernando: SymPy +1
- Thomas: Pandas +1; Statsmodels +½

Thanks,
Thomas

[Travis Oliphant]

SymPy +1

I think that should be the basic level for now --- just enough to get started.    To that, we add another level that includes

Pandas, Statsmodels, sklearn, skimage, pytables, h5py, networkx

Those two levels would be enough to define the standard.    My hesitation in adding other packages to the lowest-level core is that we should give some of the interfaces time to mature before standardizing.    There should be a "name" for this fundamental core as well. 

Best,

-Travis

[Thomas Kluyver]

On Tuesday, 2 October 2012 15:19:31 UTC+1, Travis wrote

I think that should be the basic level for now --- just enough to get started.    To that, we add another level that includes

Again, this is what I want to steer clear of: defining several 'levels' to satisfy our notions of how 'core' different projects are. If there are to be multiple levels, we need a clear rationale for why the user wants one or another. This was the key to Fernando's earlier suggestion of a second level that requires a compiler: you can do lots of interesting stuff with it, but it might be that much more complex to install.

'Getting started' is not a good user story. University internet connections are fast, and hard drives are large. Why shouldn't beginners just download the more complete stack and use only the parts they need? It's bound to be much easier than downloading the 'core' and adding to it.

I also think we need to determine the specification based mainly on functionality, with maturity as a secondary concern. I've been convinced for some months that the DataFrame functionality in pandas is an essential tool - having tried to write something similar myself, and abandoned it when I found more mature projects. I'd hate to see it excluded just because it doesn't have the long history of projects like numpy.

Thomas

[Almar Klein]

 

I think that defining a relatively basic core stack is a good idea. However, I have my doubts about defining a 'full stack'. The reason is that it would be very difficult to define one that is both complete and not too discipline-specific. It will either be incomplete or too large. Instead I propose to define multiple categories for the different disciplines. What packages go in these categories can be specified by people from these fields themselves. I've heard the R uses something like this as well.

To be clear, I don't think we ever want to define a 'full' stack, in the sense of 'this is all you need'. But at the same time, I'd hate for a newcomer's first experience with the stack to be wading into the Python Packaging Mess. The base should allow people to do useful work, rather than being a 'core' in the sense of the bare minimum. It should also allow you to share code that 'runs on the Scipy stack'.

...

- "Core packages": Python, Numpy, Scipy, Matplotlib, IPython

 

So if I understand correctly, you're saying that the scipy-stack level 0 is intended to meet some scientists demands, or at least give them something to get started with. Scipy-stack level 1 will be a richer environment, but anyone with specific needs will have to install additional packages himself. That's acceptable, as I think most these additional packages will be easy to install in general.

Nevertheless, I foresee that in some fields that means new users will need to install some special package to get started. As an example, someone wanting to do work with medical data probably needs pydicom.

 

But I'd advise against returning to a minimalist 'core' standard that everyone will want to add bits to.

 

I'm not saying that the core should be minimal.The common component can include skimage sklearn, etc. But where do you draw the line? No matter how large you make the scipy-stack, there will *always* be specific needs depending on the field of science.

I understand that an approach as I proposed makes defining a standard much harder, because there will be different sets of packages to take into account, so let's just keep it in the back of our heads for now. Maybe when packaging Just Works, something like this becomes straightforward.

  Almar

[Thomas Kluyver]

I just re-read this:

On Monday, 1 October 2012 01:49:32 UTC+1, Fernando Perez wrote:

I had drawn a line between the 'core' packages that was basically EPD
free + sympy, and the layer below is what I consider a reasonable
'full stack' that is widely applicable, stands well against Matlab and
R even with add-ons, and yet is not too discipline-specific.

Fernando's description of the 'full stack' is what I'm after. Something that's useful to lots of people without being subject-specific, and can seriously go up against competing environments. From my background, I'm somewhat familiar with R - hence why I see pandas as essential.

Drawing packages on different levels like this is a useful way to look at the ecosystem, but I don't think it's a helpful distinction to present to new users, or to codify in a standard. It relates to the age of the projects and the people behind them, not what they do.

Thomas

[Almar Klein]

Again, this is what I want to steer clear of: defining several 'levels' to satisfy our notions of how 'core' different projects are. If there are to be multiple levels, we need a clear rationale for why the user wants one or another. This was the key to Fernando's earlier suggestion of a second level that requires a compiler: you can do lots of interesting stuff with it, but it might be that much more complex to install.

So are you saying that the main reason to define the two levels is for the compiler? But isn't Cython the only package that actually requires the user to have one? If so, I'd say we go with the "full" stack from the start. Users can then use skimage etc, but need to get a compiler if they want to use Cython.

Something else for our list-of-things-to-do-later: I'd say we need to do some work to make "getting a compiler" much easier. I think much of it "being hard" is that it's unclear what exactly you need and where to get it. We can probably fix that. It can be just a link with installation instructions.

I also think we need to determine the specification based mainly on functionality, with maturity as a secondary concern.

The problem with that is that with an immature project you don't know whether it will survive time. On the other hand, being in the standard might make it do so.

Regards,

  Almar

[Thomas Kluyver]

> I'm not saying that the core should be minimal.The common component can include skimage sklearn, etc. But where do you draw the line? No
> matter how large you make the scipy-stack, there will *always* be specific needs depending on the field of science.

It's certainly not easy to draw the line - hence the dozens of posts on the subject. But I think it is worth deciding.

I like your idea to have sets of packages for particular fields, but I think it's up to the communities in those fields to define them - whether they want to do that as a standard, a distribution*, or a large central package (like BioPython).

* Another long-term project: a simple way to select components and build a complete Python distribution that you can distribute, so e.g. a university can give their students a customised distribution.

On Tuesday, 2 October 2012 16:24:58 UTC+1, Almar Klein wrote:

Something else for our list-of-things-to-do-later: I'd say we need to do some work to make "getting a compiler" much easier. I think much of it "being hard" is that it's unclear what exactly you need and where to get it. We can probably fix that. It can be just a link with installation instructions.

As I understand it, the situation looks something like this:
- Linux: it's in distribution repositories, if not already installed.
- Mac: XCode. Conceptually easy, but the >1GB download is offputting.
- Windows: Choose between VS (making sure to get the right version for your Python installation) or MinGW (which I know ~nothing about).

None of it's terribly hard for the user, but there isn't a good way to include it in a distribution, or automatically install it. Well, except on Linux, where you can just depend on the relevant package. SAGE's custom Linux VM starts to look like a good way forwards.

Thomas

[Jason Grout]

On 10/2/12 10:57 AM, Thomas Kluyver wrote:
> - Mac: XCode. Conceptually easy, but the >1GB download is offputting.

You might just be able to install the command-line tools, which is a
separate download that is on the order of 150-200M, IIRC. For example,
see http://rys.sommefeldt.com/2012/02/22/OS-X-command-line-tools.html or
http://kennethreitz.com/xcode-gcc-and-homebrew.html for discussion.

Thanks,

Jason

[Jason Grout]

I just checked. Apple has versions for Lion and Mountain Lion. The
downloads are between 110-136M. The description says it installs
command line developer tools, as well as Mac OS X SDK frameworks and
headers. Included are Apple LLVM compiler, linker and Make.

Thanks,

Jason

[Brian Granger]

I too am +1 on SymPy in the base list.

> - Thomas: Pandas +1; Statsmodels +½

Pandas +1/2 as it is useful to a large fraction of people.
Statsmodels is too specific to be in the base list.

> Thanks,
> Thomas
>
> --
>
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgra...@calpoly.edu and elli...@gmail.com

[Brian Granger]

I do think that a good survey on the package list would solicit more
feedback from the community and it would come in a form that is more
conducive to reaching an actionable result.

[Eric Firing]

On 2012/10/02 5:09 AM, Thomas Kluyver wrote:
> I just re-read this:
>
> On Monday, 1 October 2012 01:49:32 UTC+1, Fernando Perez wrote:
>
> I had drawn a line between the 'core' packages that was basically EPD
> free + sympy, and the layer below is what I consider a reasonable
> 'full stack' that is widely applicable, stands well against Matlab and
> R even with add-ons, and yet is not too discipline-specific.
>
>
> Fernando's description of the 'full stack' is what I'm after. Something
> that's useful to lots of people without being subject-specific, and can
> seriously go up against competing environments. From my background, I'm
> somewhat familiar with R - hence why I see pandas as essential.

For meteorology and oceanography, mpl-toolkits.basemap is essential, and
netCDF4 may be as well. So, do they get included? Or are they too
subject-specific? And how should this be decided?

Eric

>
> Drawing packages on different levels like this is a useful way to look
> at the ecosystem, but I don't think it's a helpful distinction to
> present to new users, or to codify in a standard. It relates to the age
> of the projects and the people behind them, not what they do.
>
> Thomas
>

> --
>
>

[Almar Klein]

* Another long-term project: a simple way to select components and build a complete Python distribution that you can distribute, so e.g. a university can give their students a customised distribution.

That has crossed my mind as well, but even better would be (when packaging improves) that a customized distribution can consist of just a 'normal' distributions plus a file that specifies additional packages to be installed...

 

On Tuesday, 2 October 2012 16:24:58 UTC+1, Almar Klein wrote:

Something else for our list-of-things-to-do-later: I'd say we need to do some work to make "getting a compiler" much easier. I think much of it "being hard" is that it's unclear what exactly you need and where to get it. We can probably fix that. It can be just a link with installation instructions.

As I understand it, the situation looks something like this:
- Linux: it's in distribution repositories, if not already installed.
- Mac: XCode. Conceptually easy, but the >1GB download is offputting.
- Windows: Choose between VS (making sure to get the right version for your Python installation) or MinGW (which I know ~nothing about).

None of it's terribly hard for the user, but there isn't a good way to include it in a distribution, or automatically install it. Well, except on Linux, where you can just depend on the relevant package. SAGE's custom Linux VM starts to look like a good way forwards.

And that's ok. A clear description on scipy.org for each OS how to obtain a compiler would already be a huge improvement.

Regards,

  Almar

[Thomas Kluyver]

On Tuesday, 2 October 2012 17:27:11 UTC+1, ellisonbg wrote:

I do think that a good survey on the package list would solicit more
feedback from the community and it would come in a form that is more
conducive to reaching an actionable result.

I've put 11 packages that we're discussing into a Doodle poll:
http://www.doodle.com/ma6rnpnbfc6wivu9

Python, numpy, scipy, matplotlib and IPython are assumed to be included already.

I'll post this to the scipy-user list as well.

Thanks,
Thomas

[Jason Grout]

Just to be clear how to interpret my answer, I treated the Ifneedbe
option as "no preference, or I don't know enough to make an informed
choice."

Thanks,

Jason

[Scott Collis]

Ditto

Sent from an iPad, please forgive my fat fingers...

----
Dr Scott Collis
ARM Precipitation Radar Translator

ARM Climate Research Facility (ACRF)
Computing, Environment, and Life Sciences Directorate
Argonne National Laboratory
Cell: +1 630 235 8025
Office: +1
630 252 0550
http://radar.arm.gov

> --
>
>