User Survey

[maldun.fin...@gmail.com]

Sorry for double posting, but I just become aware of this list, and maybe it belongs better here:

Hi!

Since William's statement, that Sage failed as a real alternative to the 4 MMs there are currently some threads
with thoughts on improving Sage.

But till now I see only discussions among the devlopers. But I think we should also ask the users.
The most of us here are scientists. But to make Sage successful we have to be more considerate
about a big block of non mathematicians and beginners, which are a big portion of potential users.

I don't think that the functionality of Sage is the big problem, in fact Sage has a great features for zero cost.
A bigger problem is that Sage is lacking good 'user experience'. This starts already with installation.
We still don't have a good Windows version, and you can't install Sage from the standard repos of your favorite distribution.
SageMathCloud overcomes some of these problems by providing
an out of the box we interface, but there are still people who want something to install on their hard drive.
Especially If they don't want to go online for security reasons, or want to use their own hardware.
Additionally it often appears to me that sage lacks of clean design.

So I ask:

Are there any serious attempts to analyze standard user needs more systematically?
(And I don't talk about bug reports) I didn't find any except for a survey from 2009.

I think it is very important to get at least some clues about
What do most need people for their daily needs
How well does these standard tasks work.
I give an example from my personal experience: Many people have
purely numerical tasks with little symbolics involved (classical in engineering)
so they will use much of numpy functionality.
If they use Sage then they often will get annoyed by the preparser not handling numpy/scipy well.
I know at least 3 people, which switched to IPython+Sympy because of that reason. Not because
Sage isn't worse, but some things don't go along very well.

A keen Sage user now would simply turn the preparser off. And that would be the answer.
This may seem quite trivial to developers which work on far harder topics and are good programmers.
Personally I don't have problems tinkering around a bit.

But 'normal' people will
not read the docu, simply want that it works out of the box
not ask for support
Most users simply want to input some formulas and want the problem be solved quite elegantly,
The feeling is very important.

So what can we do? We have to ask the users!

In the information age it shouldn't be such a big deal to make some standard surveys on SageMathCloud
and Sagemath.org. Of course a short one.

Another good start would be make some simple standard tasks as an excercise sheet for students which don't know Sage
(e.g. first training session in a math course using sage). Then let the students solve them on their own, and finally let them fill
in a form, what they liked and what not, what was difficult, what easy.

I know the most of the developers are scientists, and don't care much about good 'marketing'.
But think of Henry Ford: He didn't built the best cars the automotif sector t his time, but he gave people
what they needed, and hence was successful.

I hope I made my concerns clear.

Best regards,
Stefan

[Harald Schilly]

On Fri, Dec 5, 2014 at 9:17 PM, <maldun.fin...@gmail.com> wrote:
> But I think we should also ask the users.

I second that ;-)

> Are there any serious attempts to analyze standard user needs more systematically?
> (And I don't talk about bug reports) I didn't find any except for a survey from 2009.

I did several surveys, and well, some aggregated data was published
here -- too long ago to be of relevance. I planned to publish more
after that last study, but unfortunately some participants were
against publishing anything because they feared that some private
information can leak. Even from aggregated information.
That's very unfortunate, because they obviously didn't consider this
when they read the disclaimer in the beginning … it's also not
motivating at all to continue this. In particular, how to make sure to
be able to publish the anonymous data without vocal objections from
the community.

More than year ago I've designed a rather lengthy survey for SMC,
which is of course about Sage but also more general. It's still stuck
in nowhere and I don't know much about it's current status.

> Many people have
> purely numerical tasks with little symbolics involved (classical in engineering)
> so they will use much of numpy functionality.
> If they use Sage then they often will get annoyed by the preparser not handling numpy/scipy well.
> I know at least 3 people, which switched to IPython+Sympy because of that reason. Not because
> Sage isn't worse, but some things don't go along very well.

I'm personally fully aware of that. Regarding the Sage Notebook, I
think the pure Python mode should help them. Regarding SMC, I've
suggested William to implement a pure python worksheet (same as the
*.sagews files but called *.pyws. The benefit is that the real-time
synchronization works much better than with ipython files)
(and yes, I know that there is `%default_mode python`, but this cannot
be discovered by reading those obscure errors obtained by the
interfering "Integer" objects)

> Personally I don't have problems tinkering around a bit.

I think, the majority doesn't want to tinker around. I call this the
"white canvas" problem, maybe I've picked this up from somewhere else.
Working on a way to offer functions or small code templates from a
discoverable repository (and I don't mean the documentation) would be
a great first step.

> Of course a short one.

Regarding survey length, I think there is no problem with having
longer ones. Short ones answered by those who have no time are
probably useless anyway.

> Then let the students solve them on their own, and finally let them fill
> in a form, what they liked and what not, what was difficult, what easy.

This sounds like a great idea for a usability study. I could think
about this even more extreme: put some developers behind a sound-proof
one-way mirror and let them watch the helpless students struggle with
the product the other side. Emphasis on sound-proof ;-)

> I hope I made my concerns clear.

Willing to work on a new round of Sage survey?

-- Harald

[maldun.fin...@gmail.com]

On Friday, December 5, 2014 10:28:40 PM UTC+1, Harald Schilly wrote:

On Fri, Dec 5, 2014 at 9:17 PM,  <maldun.fin...@gmail.com> wrote:
> But I think we should also ask the users.

I did several surveys, and well, some aggregated data was published
here -- too long ago to be of relevance. I planned to publish more
after that last study, but unfortunately some participants were
against publishing anything because they feared that some private
information can leak. Even from aggregated information.
That's very unfortunate, because they obviously didn't consider this
when they read the disclaimer in the beginning … it's also not
motivating at all to continue this. In particular, how to make sure to
be able to publish the anonymous data without vocal objections from
the community.

More than year ago I've designed a rather lengthy survey for SMC,
which is of course about Sage but also more general. It's still stuck
in nowhere and I don't know much about it's current status.

Ok, I already thought that there are attempts, but I couldn't find any information till now.

What about Facebook surveys? If they are public from the beginning, there are maybe less concerns,

especially since FB isn't taken that seriously by the most people. The result is of course not that scientific, as

the data from a professional survey, but could already quite informative.

 

> Many people have
> purely numerical tasks with little symbolics involved (classical in engineering)
> so they will use much of numpy functionality.
> If they use Sage then they often will get annoyed by the preparser not handling numpy/scipy well.
> I know at least 3 people, which switched to IPython+Sympy because of that reason. Not because
> Sage isn't worse, but some things don't go along very well.

I'm personally fully aware of that. Regarding the Sage Notebook, I
think the pure Python mode should help them. Regarding SMC, I've
suggested William to implement a pure python worksheet (same as the
*.sagews files but called *.pyws. The benefit is that the real-time
synchronization works much better than with ipython files)
(and yes, I know that there is `%default_mode python`, but this cannot
be discovered by reading those obscure errors obtained by the
interfering "Integer" objects)

> Personally I don't have problems tinkering around a bit.

I think, the majority doesn't want to tinker around. I call this the
"white canvas" problem, maybe I've picked this up from somewhere else.

I'm currently reading the book 'Die Feinde des neuen' from Gunther Dück (maybe you know him; he's Professor of mathematics

and manager at IBM). It is very enlightening (at least for me) at the process how innovation 'happens'.

From innovation theory there are four kind of people:

• Protagonists
• Open Minds
• Closed Minds
• Antagonists

nicely normal distributed like in this graphics: http://regital.de/wp-content/uploads/2013/05/Ohne-Titel-9.jpg

The people who like to tinker around are part of the Protagonists sector, and unfortuanetly the smallest one.

The sector we want to reach are the open mind and then the closed mind sector. 

But already the open minds are quite pragmatic, and they don't want to waste time, in contrary to the protagonists. 

I think this explains very well, why Sage's growth has declined: We don't come across this border.

Many other projects died, because they weren't aware of these facts. 

If you have some time you could watch Dück's speach on that topic (From your name and location I may assume you speak german ;) :

https://www.youtube.com/watch?v=Ta5KutPE3U0 

He tackles many of the problems we are currently facing, and maybe could

give some new input.

Working on a way to offer functions or small code templates from a
discoverable repository (and I don't mean the documentation) would be
a great first step.

What about example worksheets with good documentation? Many text processing

programs like M$ Word or LaTeX editors like Kile give standard templates at startup.

One could implement in SMC and SageNB an option when creating a new Worksheet.

Another thing would be a snippet tool like emacs has:

https://www.youtube.com/watch?v=vOj7btx3ATg (approx starting from 1:15)

It comes along with scrollbar containing the commands.

Alternatively one could make a button bar with the most common commands on mouse click,

which generate the snippet on the current command line like Mathematica or many LaTeX editors do.

 

> Of course a short one.

Regarding survey length, I think there is no problem with having
longer ones. Short ones answered by those who have no time are
probably useless anyway.

Ok you have a point. 

> Then let the students solve them on their own, and finally let them fill
> in a form, what they liked and what not, what was difficult, what easy.

This sounds like a great idea for a usability study. I could think
about this even more extreme: put some developers behind a sound-proof
one-way mirror and let them watch the helpless students struggle with
the product the other side. Emphasis on sound-proof ;-)

The only problem would to be find such a room =D

Another step further: Let them tackle the same problems

with several programs. Then you also get some comparison date.

 

> I hope I made my concerns clear.

Willing to work on a new round of Sage survey?

-- Harald

I would be glad to help with all what is within my time and possibility scope.

- Stefan 

[William Stein]

On Fri, Dec 5, 2014 at 3:02 PM, <maldun.fin...@gmail.com> wrote:
>
>
> On Friday, December 5, 2014 10:28:40 PM UTC+1, Harald Schilly wrote:
>>
>> On Fri, Dec 5, 2014 at 9:17 PM, <maldun.fin...@gmail.com> wrote:
>> > But I think we should also ask the users.
>>
>> I did several surveys, and well, some aggregated data was published
>> here -- too long ago to be of relevance. I planned to publish more
>> after that last study, but unfortunately some participants were
>> against publishing anything because they feared that some private
>> information can leak. Even from aggregated information.
>> That's very unfortunate, because they obviously didn't consider this

>> when they read the disclaimer in the beginning ... it's also not

>> motivating at all to continue this. In particular, how to make sure to
>> be able to publish the anonymous data without vocal objections from
>> the community.
>>
>> More than year ago I've designed a rather lengthy survey for SMC,
>> which is of course about Sage but also more general. It's still stuck
>> in nowhere and I don't know much about it's current status.

It's stuck in nowhere. Why don't we try to get some people -- a small
group of maybe 10 -- to try it, and see how it works, and then get way
more people to try it?

> Ok, I already thought that there are attempts, but I couldn't find any
> information till now.
>
> What about Facebook surveys? If they are public from the beginning, there
> are maybe less concerns,
> especially since FB isn't taken that seriously by the most people. The
> result is of course not that scientific, as
> the data from a professional survey, but could already quite informative.

That's a great idea. Being "scientific" is only critical for people
who are trying to publish papers in journals, which we are not.
If all you want is ideas about what people think, it's still useful to
just talk to people informally.

> But already the open minds are quite pragmatic, and they don't want to waste
> time, in contrary to the protagonists.
> I think this explains very well, why Sage's growth has declined: We don't
> come across this border.
> Many other projects died, because they weren't aware of these facts.

Yes. Note: Sage's growth hasn't just declined. It's been at zero
growth for years.

>> Working on a way to offer functions or small code templates from a
>> discoverable repository (and I don't mean the documentation) would be
>> a great first step.
>>
>>
>
> What about example worksheets with good documentation? Many text processing
> programs like M$ Word or LaTeX editors like Kile give standard templates at
> startup.
> One could implement in SMC and SageNB an option when creating a new
> Worksheet.
>
> Another thing would be a snippet tool like emacs has:
> https://www.youtube.com/watch?v=vOj7btx3ATg (approx starting from 1:15)
> It comes along with scrollbar containing the commands.
>
> Alternatively one could make a button bar with the most common commands on
> mouse click,
> which generate the snippet on the current command line like Mathematica or
> many LaTeX editors do.

My impression is that Harald very much wants to do everything you're
suggesting above. :-)

William

>> This sounds like a great idea for a usability study. I could think
>> about this even more extreme: put some developers behind a sound-proof
>> one-way mirror and let them watch the helpless students struggle with
>> the product the other side. Emphasis on sound-proof ;-)

> The only problem would to be find such a room =D
> Another step further: Let them tackle the same problems
> with several programs. Then you also get some comparison date.

I have watched people a lot using Sage for 10 years now, and learned a
great deal from the experience. For example, several iterations
(=massive coding sprint work) of the Sage notebook were directly
targeted at using the Sage notebook in a computer-lab based course,
and I would walk around for hours as the students worked, watching
them and answering questions.

With web-based applications it's possible to statistically observe a
large number of users behavior and learn from it in a similar way.
This is a massive potential advantage of web-based applications over
traditional software...

-- William

>
>>
>> > I hope I made my concerns clear.
>>
>> Willing to work on a new round of Sage survey?
>>
>> -- Harald
>
>
> I would be glad to help with all what is within my time and possibility
> scope.
>
> - Stefan
>

> --
> --
> You received this message because you are subscribed to the Google group
> "sage-marketing".
> To post to this group, send email to sage-ma...@googlegroups.com
> To unsubscribe from this group, send email to
> sage-marketin...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-marketing
> More about Sage: http://www.sagemath.org
> ---
> You received this message because you are subscribed to the Google Groups
> "sage-marketing" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-marketin...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[maldun.fin...@gmail.com]

I have that impression too =D

But I'm seriously thinking what I can do and what I can't do to improve Sage.

What I can't do is to do much surveying. Since my Graduation I have a full time

job at a company, and currently don't do teaching any more. So I don't have

access to students with whom I could work with.

What I actually can do is writing templates, snippets, and improve documentation for beginners and the notebook.

Then get feedback from people who are close to me and working with different simple math topics,

to improve these features, and share them with the community.

 

>> This sounds like a great idea for a usability study. I could think
>> about this even more extreme: put some developers behind a sound-proof
>> one-way mirror and let them watch the helpless students struggle with
>> the product the other side. Emphasis on sound-proof ;-)

> The only problem would to be find such a room =D
> Another step further: Let them tackle the same problems
> with several programs. Then you also get some comparison date.

I have watched people a lot using Sage for 10 years now, and learned a
great deal from the experience. For example, several iterations
(=massive coding sprint work) of the Sage notebook were directly
targeted at  using the Sage notebook in a computer-lab based course,
and I would walk around for hours as the students worked, watching
them and answering questions.

With web-based applications it's possible to statistically observe a
large number of users behavior and learn from it in a similar way.
This is a massive potential advantage of web-based applications over
traditional software...

 -- William

You could also make surveys on SMC giving some small extras for doing them

e.g. Badges. Such Badges could also be given for helping other users, or writing

tutorials for others. Many other platforms like Stack Exchange are doing this to motivate it's users.

On another note: I'm currently studying japanese and I frequently use http://jisho.org/
They had a nice idea: To test out new features and design they have a subdomain

http://beta.jisho.org/ whith a new polished version including new features under

testing.

This is quite nice for users, because to people who want to use these new features

can already try out the new things, while the more conservative users can stay with

the old for the meanwhile till the new version is tested enough for regular use.

Maybe one could consider something like that for SMC too, to try out new features like the latest Sage version

before generally upgrading the whole system.

- Stefan