ANN: pandas 0.11.0 released!

[Wes McKinney]

hi all,

We've released pandas 0.11.0, a big release that span 3 months of
continuous development, led primarily by the intrepid Jeff Reback
and y-p. The release brings many new features, performance and
API improvements, bug fixes, and other goodies.

Some highlights:

- New precision indexing fields loc, iloc, at, and iat, to reduce
occasional ambiguity in the catch-all hitherto ix method.
- Expanded support for NumPy data types in DataFrame
- NumExpr integration to accelerate various operator evaluation
- New Cookbook and 10 minutes to pandas pages in the documentation by
Jeff Reback
- Improved DataFrame to CSV exporting performance
- Experimental "rplot" branch with faceted plots with matplotlib
merged and open for community hacking

Source archives and Windows installers are on PyPI. Thanks to all
who contributed to this release, especially Jeff and y-p.

What's new: http://pandas.pydata.org/pandas-docs/stable/whatsnew.html
Installers: http://pypi.python.org/pypi/pandas

$ git log v0.10.1..v0.11.0 --pretty=format:%aN | sort | uniq -c | sort -rn
308 y-p
279 jreback
85 Vytautas Jancauskas
74 Wes McKinney
25 Stephen Lin
22 Andy Hayden
19 Chang She
13 Wouter Overmeire
8 Spencer Lyon
6 Phillip Cloud
6 Nicholaus E. Halecky
5 Thierry Moisan
5 Skipper Seabold
4 waitingkuo
4 Loïc Estève
4 Jeff Reback
4 Garrett Drapala
4 Alvaro Tejero-Cantero
3 lexual
3 Dražen Lučanin
3 dieterv77
3 dengemann
3 Dan Birken
3 Adam Greenhall
2 Will Furnass
2 Vytautas Jančauskas
2 Robert Gieseke
2 Peter Prettenhofer
2 Jonathan Chambers
2 Dieter Vandenbussche
2 Damien Garaud
2 Christopher Whelan
2 Chapman Siu
2 Brad Buran
1 vytas
1 Tim Akinbo
1 Thomas Kluyver
1 thauck
1 stephenwlin
1 K.-Michael Aye
1 Karmel Allison
1 Jeremy Wagner
1 James Casbon
1 Illia Polosukhin
1 Dražen Lučanin
1 davidjameshumphreys
1 Dan Davison
1 Chris Withers
1 Christian Geier
1 anomrake

Happy data hacking!

- Wes

What is it
==========
pandas is a Python package providing fast, flexible, and
expressive data structures designed to make working with
relational, time series, or any other kind of labeled data both
easy and intuitive. It aims to be the fundamental high-level
building block for doing practical, real world data analysis in
Python.

Links
=====
Release Notes: http://github.com/pydata/pandas/blob/master/RELEASE.rst
Documentation: http://pandas.pydata.org
Installers: http://pypi.python.org/pypi/pandas
Code Repository: http://github.com/pydata/pandas
Mailing List: http://groups.google.com/group/pydata

[lexual]

Congrats to all (including myself, first release that includes one of my patches!).

I think some of the documentation work is an incredible improvement, especially the "10 minutes to Pandas" work. 

[Adam Hughes]

Is there any other documentation specific to rplot that i may be overlooking

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Wes McKinney]

They're here:

http://pandas.pydata.org/pandas-docs/stable/rplot.html

Something is wrong with how the docs are rendering, though, for some
reason. I didn't have time to investigate.

- Wes

[Neal Becker]

Did I read correctly that pandas can take advantage of numexpr?

Do I need to do anything during installation, or is numexpr auto-detected at runtime?

[Jeff Reback]

if its installed it will use it for some operations (nice accelerations for large ish operations, over say. 20k rows)

where, boolean and simple numerical (of same dtype)

eg df + df2

we are going to add more sophisticated stuff

like

df.eval('df + df2 * s') eg a complex operation that will have even more accelerated (this will of course work now but Numexpr will only see smaller pieces of this) 

in 0.12 (this is trickier as have to do some transforms on the expression before passing to Numexpr, eg alignment)

[Jeff Reback]

you need to install Numexpr (pandas does not install it)

but once installed it will use it

I can be reached on my cell 917-971-6387

[Francesc Alted]

On Wed, Apr 24, 2013 at 3:13 PM, Jeff Reback <jeffr...@gmail.com> wrote:

if its installed it will use it for some operations (nice accelerations for large ish operations, over say. 20k rows)

where, boolean and simple numerical (of same dtype)

eg df + df2

we are going to add more sophisticated stuff

like

df.eval('df + df2 * s') eg a complex operation that will have even more accelerated (this will of course work now but Numexpr will only see smaller pieces of this) 

in 0.12 (this is trickier as have to do some transforms on the expression before passing to Numexpr, eg alignment)

What do you exactly mean by alignment?  Numexpr can deal with unaligned arrays pretty efficiently.  Although probably you are talking about other kind of alignment...

Francesc

 

[Jeff Reback]

yes exactly

we need to align by the indicies of the passed objects first then can pass the actual computation to Numexpr

--

[Francesc Alted]

Ah, got it. Thanks for the explanation.

On 4/24/13 3:36 PM, Jeff Reback wrote:
> yes exactly
>
> we need to align by the indicies of the passed objects first then can
> pass the actual computation to Numexpr
>
>
> On Apr 24, 2013, at 9:25 AM, Francesc Alted <fran...@continuum.io

> <mailto:fran...@continuum.io>> wrote:
>
>> On Wed, Apr 24, 2013 at 3:13 PM, Jeff Reback <jeffr...@gmail.com
>> <mailto:jeffr...@gmail.com>> wrote:
>>
>> if its installed it will use it for some operations (nice
>> accelerations for large ish operations, over say. 20k rows)
>>
>> where, boolean and simple numerical (of same dtype)
>>
>> eg df + df2
>>
>> we are going to add more sophisticated stuff
>>
>> like
>>
>> df.eval('df + df2 * s') eg a complex operation that will have
>> even more accelerated (this will of course work now but Numexpr
>> will only see smaller pieces of this)
>>
>> in 0.12 (this is trickier as have to do some transforms on the
>> expression before passing to Numexpr, eg alignment)
>>
>>
>> What do you exactly mean by alignment? Numexpr can deal with
>> unaligned arrays pretty efficiently. Although probably you are
>> talking about other kind of alignment...
>>
>> Francesc
>> --
>> You received this message because you are subscribed to the Google
>> Groups "PyData" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to pydata+un...@googlegroups.com

>> <mailto:pydata+un...@googlegroups.com>.

>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
> --
> You received this message because you are subscribed to the Google
> Groups "PyData" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pydata+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
Francesc Alted

[Yaroslav Halchenko]

Congrats on the release!

I have uploaded 0.11.0 builds to Debian experimental and NeuroDebian.

Now it would just be needed to address failures on exotic and big endian
platforms ;)
https://buildd.debian.org/status/package.php?p=pandas&suite=experimental And as
soon as wheezy releases, I will upload 0.11.0 (with recommends on numexpr) to
Debian unstable.

Cheers,

--
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

[Tim Michelsen]

> Now it would just be needed to address failures on exotic and big endian
> platforms ;)
> https://buildd.debian.org/status/package.php?p=pandas&suite=experimental And as
> soon as wheezy releases, I will upload 0.11.0 (with recommends on numexpr) to
> Debian unstable.

Packages added to Ubuntu PPA:
http://bazaar.launchpad.net/~pythonxy/pythonxy/pandas_debian_current/revision/25

So you can install python-pandas and it should automatically install
python-numexpr, python-bottleneck

at least from tomorrow onwards for thos who have also neurodebian updated.
the PPA python-pandas already uses version 0.12 as it's build daily from
github head.

Bottleneck is here:
https://code.launchpad.net/~pythonxy/+archive/pythonxy-devel/+packages?field.name_filter=bottleneck&field.status_filter=published&field.series_filter=

Enjoy!

[Yaroslav Halchenko]

On Wed, 24 Apr 2013, Tim Michelsen wrote:

> Bottleneck is here:
> https://code.launchpad.net/~pythonxy/+archive/pythonxy-devel/+packages?field.name_filter=bottleneck&field.status_filter=published&field.series_filter=

cool -- would you be interested to finish up the packaging for upload to
mainline Debian (and thus Ubuntu)?

[Tim Michelsen]

>> Bottleneck is here:
>> https://code.launchpad.net/~pythonxy/+archive/pythonxy-devel/+packages?field.name_filter=bottleneck&field.status_filter=published&field.series_filter=
>
> cool -- would you be interested to finish up the packaging for upload to
> mainline Debian (and thus Ubuntu)?

What is to be done?

[Yaroslav Halchenko]

generate "proper" source package so it is 'lintian free': doesn't have
spurious executable non scripts under debian/, debian/copyright better
follows machine-readable
http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
, long description better be more descriptive (e.g. list most prominent
function) and should not have any warnings ;), should build from .pyx
sources thus build depend on cython (we can discuss later what to do for
backports),
RFP
http://bugs.debian.org/%23706089
should be retitled to ITP (intent to package) and should be closed in
the new changelog entry (0.7.0-1 version).

debian/copyright should list all copyright/licenses, e.g
./bottleneck/src/move/csrc/move_median.c: BSD (2 clause)
[Copyright: HOLDERS AND CONTRIBUTORS / 2011 J. David Lee. All rights reserved]

[Tim Michelsen]

>> > What is to be done?
> generate "proper" source package so it is 'lintian free': doesn't have
> spurious executable non scripts under debian/, debian/copyright better
> follows machine-readable
> http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
> , long description better be more descriptive (e.g. list most prominent
> function) and should not have any warnings ;),

Do not know what you mean by that, but I will try to investigate once I
find the time...
See comment below, I am missing a one-stop info facility show steps to
follow until the package is considered as acceptable.

> should build from .pyx
> sources thus build depend on cython (we can discuss later what to do for
> backports),

See our discussion here:
https://github.com/kwgoodman/bottleneck/issues/54#issuecomment-16386166
So I am not sure if your comment applies here.

> RFP
> http://bugs.debian.org/%23706089
> should be retitled to ITP (intent to package) and should be closed in
> the new changelog entry (0.7.0-1 version).

OK, easy catch ;-)

> debian/copyright should list all copyright/licenses, e.g
> ./bottleneck/src/move/csrc/move_median.c: BSD (2 clause)
> [Copyright: HOLDERS AND CONTRIBUTORS / 2011 J. David Lee. All rights reserved]

As always for "true & clean" debian packages, I do never understand
where the mentor take the rules for
- "should"
- "must"
- etc.
from.

There seems to be 1Mio. scattered ideas on that. And then, for me, it
looks like an endless loop where every time a change is implemented a
new request for improvement pops up.
If could could point me at the 1-3 must reads for getting my scipy stack
python stuff accepted, then I'd be happy to try my best and help out.

Kind regards,
Timmie