Sage versus Excel (spreadsheets in general): Are my arguments correct and complete?

[Rolandb]

Dear all, I suggested to a friend living in Africa (Lomé) to use Sage instead of using Excel for semi-professional work.

Excel is great - I'm a big fan - but not for semi-professional work as you have to use multiple tables, sheets, not-that-easy-to-learn VBA et cetera.

Sage origin is mathematics, but over time it has become a great tool for normal work too (especially rural areas). The new notebook (arriving with version 5.7?) will probably boost this.

I listed the following arguments:

- Excel has a free format approach, and an (small) error is easily made
- Excel is basically two-dimensional; Sage endless
- It is difficult to combine text (explanations) plus calculations; Sage is integrated
- It is (more) difficult to reuse parts of an Excel sheet; in Sage you can define classes and therefore parts are immediately reusable
- Downloading and saving data goes better in Sage; you can more safely use linkages to external databases (with up-to-date data)
- In Excel you can use Sage...
- Advanced use of Excel requires to learn VBA; in Sage all can be done via one (easy to learn) language Python
- In Sage you can have as output formula; in Excel not.
- Finally, Sage has access to a huge set of routines, packages, classes and functions. Excel is limited.

In total: Excel is a great tool for a simple problem where you can swiftly get nice output.
For semi-professional work, Sage wins on almost all aspects IMHO. Especially when things become more complicated, Sage saves you time...

Link with a few arguments: http://iae-pedia.org/Free_Math_Software

Links which Sage easily can tackle: https://controls.engin.umich.edu/wiki/index.php/ExcelModelingGeneral and http://www.seomoz.org/blog/excel-statistics-for-seo-and-data-analysis

N.B.: The Excel example my friend was referring to: http://www.stats.ox.ac.uk/__data/assets/excel_doc/0020/8354/LifeMIPricingModuleIFOAWGv097.xlsx

Thanks in advance for your comments, additional links and experiences!

Roland

[Keshav Kini]

Rolandb <roland.va...@gmail.com> writes:

> The new notebook
> (arriving with version 5.7?) will probably boost this.

If you mean the new UI being demo'd at http://flask.sagenb.org/ , then
as far as I know there is no particular Sage version which we are
planning to release it. It should be pretty soon, though, as Samuel's
wrapping up development on it and it's in the testing phase (please do
test at the link above, if you have time, and report any bugs or
disappeared features on the sage-notebook list :) ).

-Keshav

[Emmanuel Charpentier]

Ahem ...

This :

Le jeudi 17 janvier 2013 08:35:34 UTC+1, Rolandb a écrit :

[ Bandwidth savings ... ]

 

Sage origin is mathematics, but over time it has become a great tool for normal work too (especially rural areas).

is a pearl ! I hereby nominate it as The Quote Of The Year on Mathematics...

Question : can one use Sage for abnormal work in rural areas ? or for normal work in urban areas ?

Back to my semi-abnormal work in  an overly urban area...

                                                                                   Emmanuel Charpentier

[Rolandb]

Hi, I understand that my request and language used may sounded weird. Excuses. But I still think that the impact following event could have been less if people in Africa would have access to 'normal' tools such as Sage.

Last week, during two consecutive nights (Thursday and Friday), the 2 major markets of Togo burned. In Lomé the fire started on Friday evening 23 PM and lasted until late Saturday. Ghanaian firemen(pompiers) had also to come from Aflao to help.

In Kara, 1750 shops were burned and in Lomé, 4250 shops. Victims, all informal market workers, were not insured; they lost not only their shop, their goods, but many also all their money. Unfortunately the victims left their money in the shop; they don't "trust" the local financial institutions and the big banks are not interested in them. One 'maman benz' (as they are called) lost 700 millions FCFA of goods and money. Another left her just received micro-finance loan (20 millions FCFA) in the shop. 

The Head of State, Faure Gnassingbé, called upon massive solidarity to accompany the victims of thes fires. A "catastrophe fund" will be created and all financial actors 'should' contribute.

In order to contribute, I have a kind request not for money but to provide arguments regarding my comparison between two approaches. This will help them to build a better future.

Roland

Op donderdag 17 januari 2013 14:51:15 UTC+1 schreef Emmanuel Charpentier het volgende:

[LFS]

Very badly written since I am in a hurry - IMHO:
After struggling for the last 2 weeks, I would say Excel and GeoGebra have more functions for the statistics math classroom (e.g. RandomBetween, CountIf). They are much more intuitive and better documented.
And it seems to me that Sage without numpy or other import (and thus learning) cannot do simple things like a histogram, ...(I can't even get it to do a histogram with imports.)
The trouble here is size - Sage can do much bigger and better generations of data and thus one can "show" things.

[William Stein]

stats.TimeSeries([list of values]).plot_histogram()

William

>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-support" group.
> To post to this group, send email to sage-s...@googlegroups.com.
> To unsubscribe from this group, send email to
> sage-support...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-support?hl=en.
>
>



--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[Dima Pasechnik]

On 2013-01-17, Rolandb <roland.va...@gmail.com> wrote:
> ------=_Part_13_32658695.1358408134034
> Content-Type: text/plain; charset=ISO-8859-1
> Content-Transfer-Encoding: quoted-printable
>
> Dear all, I suggested to a friend living in Africa (Lom=E9) to use Sage=20

> instead of using Excel for semi-professional work.
>

> Excel is great - I'm a big fan - but not for semi-professional work as you=
>=20

> have to use multiple tables, sheets, not-that-easy-to-learn VBA et cetera.

For serious statistics work, one can just use R (which is distributed
with Sage, by the way). http://www.r-project.org/

Instead of Excel, I'd use Google Spreadsheets.
(Currently they only work offline with Google Chrome browser, and you
need a reasonable internet connection for one-time setup (installing the
browser and enabling offile Google Drive)).
At least you don't pay any Microsoft tax then :)

[LFS]

Thank-you. Perhaps we could add that somewhere, e.g. http://modular.math.washington.edu/edu/2012/1062/projects/final/le-kofmehl/Math480_Quick_Reference_Statistics_Final.pdf
I tried everything on this page, everything on the forum, absolutely everything I could find doing a search on: sage plot histogram
BTW: I see I can add bins=5 as an argument, but I cannot find a way to create a list of bin endpoints.
ex.stats.TimeSeries(mu_list).plot_histogram(bins=5,color='red', figsize=6, aspect_ratio=2)

[Javier López Peña]

On Sunday, January 20, 2013 6:27:16 AM UTC, Dima Pasechnik wrote:

For serious statistics work, one can just use R (which is distributed
with Sage, by the way).  http://www.r-project.org/

Or if you want to keep things pythonic just install the python data analysis library with

easy_install pandas 

from the sage shell (see [1] for info on Pandas).

Perhaps we should make an experimental/optional package for this?

Cheers,
J

[1] http://pandas.pydata.org/

[LFS]

I have to admit that I did not find very much in sage for descriptive statistics.
I have put what I found in my sagemath page: http://sagemath.wikispaces.com/Statistics
If anyone can add anything to this, that would be great.
I really want sage - my goal is sage as a reasonably easy to use foss substitute for MatLab or Mathematica for students of engineering or science (who are neither experts in programming nor mathematics).
Thanks for any help.

[kcrisman]

Numpy/Scipy have some ok stats stuff which is "Pythonic" (rereading your post, I guess you know that), and there are even some tutorials out there which show a bit of them used in Sage.

https://groups.google.com/forum/?fromgroups#!topic/sage-support/O-pRTcX4tV0

also is helpful.  Oh and someday we need to review http://trac.sagemath.org/sage_trac/ticket/9671 !! :-(  But LFS, you could actually use that on a local installation.

See also the RealDistribution implementation of histograms, which however I do not recommend using.

+++

On an unrelated note, I'll put in a plug for some enterprising people to make more videos to put on LFS' sagemath YouTube channel!

[LFS]

Thanks for the reply! I used everything I found on this: http://www.sagemath.org/doc/reference/sage/stats/basic_stats.html and I had looked at 9671 and said "Argh".
 

kcrisman wrote> On an unrelated note, I'll put in a plug for some enterprising people to make more videos to put on LFS' sagemath YouTube channel!

 

Oh some new videos would be great. Simple and short. (I owe some more on statistics on geogebrachannel, but after 22 Feb, I should be free again to work on sagemath, but some videos by people who know what they are doing would be great!)
P.S. I kept forgetting the "common" password so I changed it back to one I know [old, older, ...] - For sure if anybody makes a video, I can either upload it (mp4) or write me (lfahlberg at gmail.com) and I will remake a "common" and hopefully memorable password :)

[William Stein]

On Tue, Jan 29, 2013 at 1:23 PM, LFS <lfah...@gmail.com> wrote:
> Thanks for the reply! I used everything I found on this:
> http://www.sagemath.org/doc/reference/sage/stats/basic_stats.html and I had
> looked at 9671 and said "Argh".

That is just the result of a small undergrad student project. There
is *massively* more about statistics in Sage... including scipy.stats,
which is pretty powerful, and of course all of R.

William

>
>>
>> kcrisman wrote> On an unrelated note, I'll put in a plug for some
>> enterprising people to make more videos to put on LFS' sagemath YouTube
>> channel!
>
>
> Oh some new videos would be great. Simple and short. (I owe some more on
> statistics on geogebrachannel, but after 22 Feb, I should be free again to
> work on sagemath, but some videos by people who know what they are doing
> would be great!)
> P.S. I kept forgetting the "common" password so I changed it back to one I
> know [old, older, ...] - For sure if anybody makes a video, I can either
> upload it (mp4) or write me (lfahlberg at gmail.com) and I will remake a
> "common" and hopefully memorable password :)
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-support" group.

> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-support...@googlegroups.com.

> To post to this group, send email to sage-s...@googlegroups.com.

> Visit this group at http://groups.google.com/group/sage-support?hl=en.

> For more options, visit https://groups.google.com/groups/opt_out.

[Jason Grout]

On 1/29/13 4:18 PM, William Stein wrote:
> On Tue, Jan 29, 2013 at 1:23 PM, LFS <lfah...@gmail.com> wrote:
>> Thanks for the reply! I used everything I found on this:
>> http://www.sagemath.org/doc/reference/sage/stats/basic_stats.html and I had
>> looked at 9671 and said "Argh".
>
> That is just the result of a small undergrad student project. There
> is *massively* more about statistics in Sage... including scipy.stats,
> which is pretty powerful, and of course all of R.
>

When we upgrade numpy [1], we should also seriously look at including
pandas [2]

Thanks,

Jason

[1] http://trac.sagemath.org/sage_trac/ticket/11334
[2] http://pandas.pydata.org/

[Harald Schilly]

On Tuesday, January 29, 2013 11:47:35 PM UTC+1, Jason Grout wrote:

When we upgrade numpy [1], we should also seriously look at including
pandas [2]

big +1 from me. And I would also like to see statsmodels :-)

http://statsmodels.sourceforge.net/

H 

[LFS]

Hi guys - I am sure that there is much more available. But actually William you hit the nail on the head. The page I used was "a small undergraduate student project" and its contents are useful and accessible. That is the level I want  - so a student or someone like me can use Sage in place of MatLab or Mathematica in the undergraduate mathematics classroom without getting bogged down in learning programming. To me, the problem is that so often (a) I cannot find something basic or if I do: (b) I cannot understand how to use it in Sage. Do a search for "sage statistics" or "sage histogram" and you are immediately passed off to one or more of the underlying programming languages each with its own specific syntax style. For the first search, that one page is all there is for sage. For histograms, I spent many, many hours searching before I wrote my first comment in this thread. I found and tried every single one of the commands in the "sage stat reference" pages. I did the scipy stuff. I could sort of get an array with the counts and the bins if I worked at it really hard (I had to separate out the returns), but I never could find a way to plot it. I am sure there is some reference, but I could not find it.
Perhaps this: "a student or someone like me can use Sage in place of MatLab or Mathematica in the undergraduate mathematics classroom" is not a goal of Sage. However, my impression was that this was one of the goals and it was the reason I started to use it and work at it (and build my own wiki and youtube channel - places that explain the basic stuff). I am not complaining, but rather explaining that "massive" and "complete" is not always that helpful in the undergraduate (and even graduate) math classroom.
Finally, thanks always to everyone for replying any which way to my questions. An active forum is a blessing and one of the many excellent points of Sage.

[kcrisman]

On Wednesday, January 30, 2013 8:26:16 AM UTC-5, LFS wrote:

Hi guys - I am sure that there is much more available. But actually William you hit the nail on the head. The page I used was "a small undergraduate student project" and its contents are useful and accessible. That is the level I want  - so a student or someone like me can use Sage in place of MatLab or Mathematica in the undergraduate mathematics classroom without getting bogged down in learning programming. To me, the problem is that so often (a) I cannot find something basic or if I do: (b) I cannot understand how to use it in Sage. Do a search for "sage statistics" or "sage histogram" and you are immediately passed off to one or more of the underlying programming languages each with its own specific syntax style. For the first search, that one page is all there is for sage. For histograms, I spent many, many hours searching before I wrote my first comment in this thread. I found and tried every single one of the commands in the "sage stat reference" pages. I did the scipy stuff. I could sort of get an array with the counts and the bins if I worked at it really hard (I had to separate out the returns), but I never could find a way to plot it. I am sure there is some reference, but I could not find it.
Perhaps this: "a student or someone like me can use Sage in place of MatLab or Mathematica in the undergraduate mathematics classroom" is not a goal of Sage. However, my impression was that this was

Unfortunately, I don't think we have enough people using Sage per se (as opposed to R or Scipy) for stats who would also have time and be willing to write exactly this worksheet!  For instance, I am not a statistician, nor have ever taken a course in stats, so our attempt http://sage.maa.org/home/pub/24/ at this (see also http://trac.sagemath.org/sage_trac/ticket/13381 where this would enter the standard doc) is unfortunately one of the better pieces.  It's a chicken-egg problem - we need people who use this stuff daily in the classroom to write such resources, but they aren't able to use Sage easily enough to be motivated to do so :(

It is SO unfortunate that the sagenb.org published worksheets had to be turned off - often a half hour of searching there yielded untold stores of fruit.

Thanks for your feedback and efforts; they really are appreciated. 

[LFS]

It is SO unfortunate that the sagenb.org published worksheets had to be turned off - often a half hour of searching there yielded untold stores of fruit.

1. It REALLY is unfortunate. I used those published worksheets ALL the time. Without them I NEVER would have continued on with Sage.
Now 2 problems: I have nowhere to go and I never downloaded the ones I did use (just referenced them).

2. Thanks so much for the link to your publication. From there and work I had done, i FINALLY got histograms to work decently with (I think) a minimum of "importing" and weirdness.
Also I will extend to other distributions that you worked there, but I wanted to get something easy up.

----
I will try to get a wikipage up, but meanwhile:  http://sage.math.canterbury.ac.nz/home/pub/255

Question for anyone: How to I get my histograms from matplotlib to be smaller?

Thanks!

[Harald Schilly]

On Thursday, January 31, 2013 12:22:53 PM UTC+1, LFS wrote:

Question for anyone: How to I get my histograms from matplotlib to be smaller?

could you please post the full command of such an example plot? i haven't followed everything and there are several ways how such a plot is actually invoked!

thanks, harald 

[LFS]

Hiya Harald - I will try. I am really, really bad with forums. That is why I posted the link to the public page. I think the version is 4.8. (Can't use sagenb since no public posting.)

Here is the minimal code version I used in an example (for people like me who don't understand all the programming involved).

my_data=[random() for j in range(100)]
import matplotlib.pyplot as plt
plt.hist(my_data, bins=5, range=(0.,1.))
plt.savefig('Histogram.png')
plt.close()

From what I understand, the last two lines are to force an actual plot.
-----------------------------------------
Here is a slightly more expanded version where I can extras to the plot, but cannot get it to be a decent size.

my_data=[random() for j in range(100)]

import matplotlib.pyplot as plt
fr, bn, junk= plt.hist(my_data, bins=5, range=(0.,1.), normed=1, facecolor='lightgreen')  #I bet everyone will appreciate my label of "patches"
print 'normed frequency array is: ', fr
print 'bins array is: ', bn

xmn=0;xmx=1;ymn=0;ymx=max(fr)+1
plt.axis([xmn,xmx,ymn,ymx])

plt.xlabel('Intervals', color=(0,.4,.4))
plt.ylabel('Frequency', color=(.4,0,.4))
plt.grid(True)

plt.savefig('Histogram2.png')
plt.close()

Hope I did this correctly. Thanks! Linda

[Harald Schilly]

On Thu, Jan 31, 2013 at 4:50 PM, LFS <lfah...@gmail.com> wrote:

Here is the minimal code version

ok, this is a straightforward matplotlib-only question. therefore the answer is easy :-)

before the histogram line, create a figure and define its size (tuple of inch values):

plt.figure(figsize=(2,2))

when you save it, you can also define a dpi value:

plt.savefig('Histogram.png', dpi=600)

both give you all the flexibility you might want.

Harald

[LFS]

Thanks Harald! That worked and I very much appreciate your quick reply.  Straightforward is what I want!

BTW: I am satisfied with this, but I will point out the following to anyone else reading this thread
   The first solution does cut off the ylabel a bit from the left; The second makes the fonts rather small - i used dpi=50 (I am sure you can fix this with font size on the labels ...)

[Harald Schilly]

On Thu, Jan 31, 2013 at 5:40 PM, LFS <lfah...@gmail.com> wrote:

The second makes the fonts rather small - i used dpi=50

dpi is the size of the raster for the bitmap, and the other size is the number of inches the image should be. fonts are rendered to fit into this, therefore they apparently change size!

http://en.wikipedia.org/wiki/Dots_per_inch

therefore, it depends on where you want to show the image…

H

[William Stein]

On Thu, Jan 31, 2013 at 3:22 AM, LFS <lfah...@gmail.com> wrote:
>
>> It is SO unfortunate that the sagenb.org published worksheets had to be
>> turned off - often a half hour of searching there yielded untold stores of
>> fruit.
>>
>
> 1. It REALLY is unfortunate. I used those published worksheets ALL the time.
> Without them I NEVER would have continued on with Sage.
> Now 2 problems: I have nowhere to go and I never downloaded the ones I did
> use (just referenced them).

In case somebody wants to somehow host them right now, I've just made
a tarball of *all* worksheets ever published worksheets on sagenb.org.
Here it is:

http://wstein.org/tmp/sagenb-pub.tar.bz2

If somebody wanted to host this somewhere, with some sort of index,
they could. There would definitely want to strip out the javascript
"malware" though -- which are in there.

I *will* be scanning and transforming all these worksheets to be
available from Salvus in the next few months. I will do this starting
with exactly the tarball posted above.

William

>
> 2. Thanks so much for the link to your publication. From there and work I
> had done, i FINALLY got histograms to work decently with (I think) a minimum
> of "importing" and weirdness.
> Also I will extend to other distributions that you worked there, but I
> wanted to get something easy up.
>
> ----
> I will try to get a wikipage up, but meanwhile:
> http://sage.math.canterbury.ac.nz/home/pub/255
>
> Question for anyone: How to I get my histograms from matplotlib to be
> smaller?
>
> Thanks!
>