Managing the user's expectation

[Ariya Hidayat]

Folks,

This is a call for discussion.

Lately, I've seen an increase in the amount of frustation of the
users. Often, they show up in various social media. The pattern is
very typical: a particular feature is not available and therefore it
triggers the frustation.

The impact of such a user being upset is varying. On the positive
side, there are sometimes offers from folks who wants to give
sponsorship to expedite a particular feature of bug fix [1]. In
addition, some critics are very direct, healthy, and constructive. Of
course, the best outcome is when someone really takes his time and
volunteers a fix, and that happens several time. Kudos!

The negative response is interesting as well. It goes from a knee-jerk
reaction to the usual passive-aggressiveness. Thanks to the wonderful
world of social media, this often propagates to other parties, which
would further cement the negative impression. This seems to be
harmless but my past years running the project show that it could be
worse that you think. A memory of negative impression somehow sticks
longer in our brain, maybe a justification of such bitterness?[2] Also
remember that as a modern Homo sapiens sapiens, we are trained to
mourn the loss of an eye even though the two legs and two arms are
still intact.

We can argue that we should just ignore all the negative reaction
("Haters gonna hate"). I would agree to that one year ago. However,
the outreach of PhantomJS today is quite different. A lot of
downstream projects depend on the its development healthiness. We also
have a growing number of contributors. If it is fine for a user to
vent his anger, what do we tell to a contributor if he is frustated?
Eventually, someone will think "Why I should waste my weekends fixing
this crash if that bloke still whine out loud?".

I assume that the same situation hits any similar projects with a few
hard-working volunteers but a large user base (some of them are bound
to be less impatient). Handling such a demand always requires a
special set of skills. The million-dollar question is, what should we
do better to manage the user's expectation?

Ideas? Feedback? Flames?


Thank you!

Best regards,


[1] Unfortunately, personally I can't take that due to many other
obligations (I had a huge family-time debt in 2012, I want to decrease
it in 2013 and not increase it further).
[2] These days, I still need to explain to some people that PhantomJS
is fully headless on Linux, a feature implemented > 9 months ago.


--
Ariya Hidayat, http://ariya.ofilabs.com
http://twitter.com/ariyahidayat
http://gplus.to/ariyahidayat

[Bryan Bishop]

On Mon, Jan 14, 2013 at 12:37 AM, Ariya Hidayat <ariya....@gmail.com> wrote:

downstream projects depend on the its development healthiness. We also
have a growing number of contributors. If it is fine for a user to
vent his anger, what do we tell to a contributor if he is frustated?

Can you elaborate on this? Exactly how many new core contributors are there? From my perspective, I see many people asking a lot of this project and very few others stepping up to the plate to deliver. But perhaps this is not accurate?

- Bryan
http://heybryan.org/
1 512 203 0507

[Ariya Hidayat]

The number of contributors is always hard to quantify. From my
subjective feeling (as a maintainer), I think these days I got more
help from a variety of people, compared to the situation one year ago.
As for the mailing-list discussion, not everyone has an abundance of
time and participate in every discussion (which is understandable).

If we want to count the Git commits, the top 5 in Q4 2012 gives the following:

53 Ariya Hidayat
26 Jon Leighton
19 Ivan De Marino
12 Juliusz Gonera
5 Jim Evans

while Q3 2012 looks like:

50 Ariya Hidayat
34 Ivan De Marino
9 Vitaliy Slobodin
6 execjosh
6 James M. Greene

Again, it's not easy to be objective there since e.g. Ivan develops
most of Ghost Driver code outside PhantomJS repository.

Regards,

[Nicolas Perriault]

On Mon, Jan 14, 2013 at 7:37 AM, Ariya Hidayat <ariya....@gmail.com> wrote:

> The million-dollar question is, what should we do better to manage the user's expectation?

I don't have a direct answer, but a possible explanation on the
frustration; typical phantomjs users are frontend & backend Web
developers. They eventually know javascript, but then often don't know
C++ at all (like me). So contributing C++ code is a bit of hard for
many users, that's probably why they feel frustration having to rely
on someone else to make things happen.

So at least if we want people to contribute more code to phantomjs, we
should lower the barrier for those people - especially C++/Qt
beginners - to get started with contributing C++ code. Well, that's
not an easy task for sure… but a «beginner's guide to Qt & phantomjs»
with eg. how to setup an IDE, where to find basic information on the
framework, how to test a patch, etc. would be a nice start

Also, what I would suggest is a better communication on the roadmap
(targeted features & bug resolution), maybe with blog entries or
regular posts on this very google group…

Last, even if the documentation is really better now than when hosted
on google code, it still needs some effort to make it complete &
accurate for users; for example, I realized yesterday than the frame
manipulation API wasn't documented on the wiki… That certainly can
create frustration :)

My two cents

--
Nicolas Perriault
https://nicolas.perriault.net/
Phone: +33 (0) 660 92 08 67

[Bryan Bishop]

On Mon, Jan 14, 2013 at 1:24 AM, Nicolas Perriault <nic...@perriault.net> wrote:

I don't have a direct answer, but a possible explanation on the
frustration; typical phantomjs users are frontend & backend Web
developers. They eventually know javascript, but then often don't know
C++ at all (like me). So contributing C++ code is a bit of hard for
many users, that's probably why they feel frustration having to rely
on someone else to make things happen.

If that was really the root reason, they would probably contribute to pyphantomjs instead, which they don't. Most backend web developers can muddle their way through python easily enough (python/django as an excellent example). Maybe there is a more complex reason involved?

[Nicolas Perriault]

On Mon, Jan 14, 2013 at 8:57 AM, Bryan Bishop <kan...@gmail.com> wrote:

> If that was really the root reason

I never claimed that

> they would probably contribute to pyphantomjs instead, which they don't.

pyphantomjs was a parallel port of the phantomjs C++ codebase to
Python, which is - if you want my opinion - quite a waste of energy.
Very few coders I know like porting & reproducing stuff like monkeys
from a language to one other.

If the phantomjs codebase were in Python, I'd probably contribute a
whole lot hell of more, for sure.

++

[Vitaliy Slobodin]

The main reason of negative user responses - broken features or bugs.

They create issues and expect that it will be fixed quickly. But they don't realizing that PhantomJS is open source.

Each developer is working on it in their free time, each developer has some limitations: resources, time, etc.

But, on the other hand, many questions arise due to the lack of documentation. Or, they think it's better to ask in the mailing list to receive a ready solution.

It's like, I have Google, why should I read the documentation? My point is to improve the documentation, FAQ  and other docs.

PhantomJS is the complicated project, especially Qt-Webkit bridge. And, as Nicolas mentioned, most of the users are web developers. They can do only one thing: use it, and report an issue. And getting started guide "How to be involved" could be very helpful. As a side note, I've already thought about - create a blog to start writing about hacking or contributing to PhantomJS. Currently PhantomJS little closed to users. Only those who are beginning hacking and has knowledge of C++ and Qt can understand what is going on inside of PhantomJS. When user coming to get our help, he often get the answer: read the doc, please.

Of course, this is a special case. James Greene is doing an amazing job here - helps users. To solve it or improve, I think we need to create a wiki page, that explains how PhantomJS works: page context, sandbox, and other things.

Thanks, Vitaliy.

---

http://lkd.to/vitallium

[Jim Evans]

I think I have a unique perspective on this issue, and hopefully a valuable one. I've been a core contributor to the Selenium project for a little over 3 years. Over that time, I've taken over ownership of the IE driver, which is also written mostly in C++. I can state pretty definitively that being a C++-based project puts up a pretty high wall between developers who can contribute, and those who might be proficient in other languages, but lack the knowledge or confidence in C++, not to mention WebKit or Qt. So there's a mismatch between users' expectations and the project developers' time-and-attention availability.

And, sadly, the problem is going to get worse. Thanks to GhostDriver, PhantomJS is starting to get attention from Selenium WebDriver users. On the whole, this is good for both projects, but there is a significant portion of WebDriver users who routinely struggle with programming concepts that I, for one, take for granted.

One thing we constantly struggle with in the Selenium project are poor issue reports. Most of our (Selenium) issues occur against a specific HTML page, without which, it's impossible to reproduce. Many WebDriver issue reporters also "fire and forget," reporting the issue without bothering to check if further information is requested. When they do check, there is often resistance to providing the required information, even though the reasons for not providing it are usually not valid[1].

I don't have a magic bullet solution. Clear communication about the volunteer nature of the development team is one thing that should help a little. Having a set release schedule is a great start too, and I'm thrilled that PhantomJS is able to make that happen on a regular basis. As for the sense of entitlement some users feel that their pet issue should be fixed, and fixed right away, I don't know how to approach that without being rude.

Docs help, but only if they're read. Anyone passionate about poor documentation should be invited and encouraged to provide docs patches. The saying "patches welcome" need not only apply to functional code, but to documentation as well.

--Jim

[1] http://jimevansmusic.blogspot.com/2012/12/not-providing-html-page-is-bogus.html

[Ivan De Marino]

And getting started guide "How to be involved" could be very helpful. As a side note, I've already thought about - create a blog to start writing about hacking or contributing to PhantomJS. Currently PhantomJS little closed to users. Only those who are beginning hacking and has knowledge of C++ and Qt can understand what is going on inside of PhantomJS. When user coming to get our help, he often get the answer: read the doc, please.

I think the idea of the "how to get involved" is really good.

We DO need more people contributing.

Yes, C++/Qt might seem not something for everyone, but I do believe that out there there are very smart individuals that are eager to absorb something new and put it to good use.

Plus, I believe this kind of stuff is like a Flywheel: slow at start but once we "bootstrap them", they can continue by themselves.

Ivan

[Ivan De Marino]

One thing we constantly struggle with in the Selenium project are poor issue reports. Most of our (Selenium) issues occur against a specific HTML page, without which, it's impossible to reproduce. Many WebDriver issue reporters also "fire and forget," reporting the issue without bothering to check if further information is requested. When they do check, there is often resistance to providing the required information, even though the reasons for not providing it are usually not valid[1].

Yes. We'd probably need to find a way to "educate" on how to report issues. 

[Ivan De Marino]

Hi guys,

In my opinion is the most pressing thing is: time between a (good) issue report, and the moment in which gets fixed.

# Choose by vote

In my opinion, the approach should be that main contributors, most of the time, pick issues to work on, based on vote.

This is something I tried to do with GhostDriver: picked the thing on top of the list.

If it wasn't that one, I would have probably started experimenting with Gecko on Qt (and you can probably predict where I'm going there...).

I have followed "what the others asked for", and I'm sticking with it (I predict, at least for another year): it's fun and I like it.

# My time, my choice

That said, I also think that contributing to Open Source MUST have an aspect of "self fulfilment and self remuneration": there is selflessness in the sharing - but the fun is selfish.

What I mean? I mean that whoever contributes should feel free to pick what she wants to work on, and spend her time on that.

IMHO, we got so far BECAUSE all the contributors worked mostly on what they liked to work on.

The "laments" of users have only partially influenced the direction.

Bear in mind, I'm not advocating to "ignore the users and do whatever you want": I'm just saying that only if we keep an healthy preference for doing what's the most fun, we will have an evolving, ever improving project.

# And idea on having new contributors

In regards of the topic "more contributors needed", we might also do a regular "get started issues collection": going through the bug tracker, the simplest, most straightforward issues get selected, and published as:

    <<If you want to contribute to PhantomJS and don't know where to start from, try those!>>

Once a "newtor" (i.e. newbie contributor) picks her battle, one of the "mator" (i.e. main contributor) can be assigned as mentor to provide guidance (without abuse).

Might start slow, but surely the result would be good for the project AND for the newtor.

My 2 cents.

I'm very curious to see where this topic will bring us :)

[James Greene]

Thanks for the kudos, Vitaliy.  I hope you plan to stick around, I'm lovin' your patches and Qt knowledge sharing! :)

I agree with pretty much everything that has been said so far:

• C++ knowledge, let alone Qt and WebKit, is a big barrier to entry for front-end web developers like myself.  That said, if they are willing to put in the effort to learn (as I did), then it is certainly not impossible.
  
  
• We need to get this documentation auto-generating from the C++ and JS codebase.  As mentioned, the API reference is still missing huge chunks of info, e.g. the frames API.  If we can just generate it from the developer's comments (e.g. JSDoc-style) and then just make doc generation a part of the [pre-commit] build, that would get us back on track from the current intermittent docs updates.  The wiki approach has also been troublesome as we have to target it to a specific version of PhantomJS (e.g. right now the API reference is targeted at 1.8).  If we instead relied on auto-generated documentation, we could have a set pertinent to the codebase at any point in the commit history (main focus on the release branches/tags, of course).
  
  We use the "docs" concepts vs. wiki for ZeroClipboard and keep our docs in Markdown format, which allows anyone to view them in a pretty format on GitHub (or elsewhere, of course) just as if it were a Wiki anyway.
  
  
• We definitely need a "Get Contributing!" guide.  I wanted badly to contribute but I spent the first week or two ping Ariya and Ivan with questions about Qt, so that would be a definite deterrent to anyone who isn't as persistent as I was.  And, of course, I'm still learning it all to this day, e.g. recently Vitaliy taught me about the Qt NAM and its [odd] request-reply model.

• I know Ariya used to work for Trolltech and that has obviously been fundamental in PhantomJS's success to date.  However, I would love to recruit 1+ devs who are actively working on each of the Qt and/or WebKit codebases who can offer us more insight and help in those particular domains for features and bugs and thus save us from needing to dig in to totally separate project codebases (which takes time away from our PhantomJS codebase work).

Sincerely,

    James Greene

--
You received this message because you are subscribed to the Google Groups "phantomjs" group.
Visit this group at http://groups.google.com/group/phantomjs?hl=en.
 
 

[Bryan Bishop]

On Mon, Jan 14, 2013 at 11:42 AM, Ivan De Marino <detron...@gmail.com> wrote:

> If it wasn't that one, I would have probably started experimenting with
> Gecko on Qt (and you can probably predict where I'm going there...).

Something like zotero's translation-server?

https://github.com/zotero/translation-server

Just wondering.

[James Greene]

(Or a headless SlimerJS?) :)

~~James

[Andrew Ray]

The best response is just to ignore the negativity in messages and respond in a timely fashion. If software is broken / buggy / crashy people will complain about it. Timely, neutral responses are all you need to calm most situations. People just want to know they're being heard and there's a light at the end of their software investment tunnel.

[Jon Leighton]

On 14/01/13 06:37, Ariya Hidayat wrote:
> Ideas? Feedback? Flames?

A few thoughts on this:

* There could be a clearer call to action for phantomjs contributors on
the website. This would obviously need some sort of "contributing guide"
wiki page to be written, highlighting the various ways people can
contribute (join the mailing list, triage bugs, write docs, submit PRs).
Any takers to write such a document?

* Negative feedback is louder than positive feedback. People usually
only make themselves heard when there's a problem, rather than when
things are going swimmingly. I'm sure there are lots of happy PhantomJS
users out there who are just getting on with things. Obviously it's
important to try to address problems/criticism, but we shouldn't assume
that these problems mean that everybody is having trouble with PhantomJS
- if anything it just means lots of people are using it.

* Regarding C++, it is a high barrier to entry, but I personally had no
idea about C++ before getting involved with PhantomJS. It can be learned
- it's certainly not an insurmountable problem to somebody who has prior
programming experience if they put their mind to it. I think a lot of
people who have come to programming via scripting languages just assume
that C/C++ is impossible for some reason.

Jon

[execjosh]

It seems that a lot of the user base is JavaScript savvy.  It also seems that, as Nicolas has pointed out, much of our implementation is in C++.  Therefore, I think the devs really should strive to implement as much as possible in JavaScript, leaving only the bear-minimum heavy lifting for C++.  Then, it might be easier for them to pinpoint exactly what's wrong (5W1H).

I also agree with writing a "how to help out" and automation of API doc generation.

[Ivan De Marino]

It seems that a lot of the user base is JavaScript savvy.  It also seems that, as Nicolas has pointed out, much of our implementation is in C++.  Therefore, I think the devs really should strive to implement as much as possible in JavaScript, leaving only the bear-minimum heavy lifting for C++.  Then, it might be easier for them to pinpoint exactly what's wrong (5W1H).

O_o

I don't want to side track, but let me say: I totally disagree with this point. Vehemently.

We can discuss this further somewhere else.

[James Greene]

While I totally agree with Josh on the point of more JS implementation leading to more active contributors, I can also agree with Ivan's stance on the matter: C++ implementations are compiled ahead of time (ergo faster and syntactically validated) whereas the JavaScript implementations are evaluated at runtime (ergo slower and possibly contains syntax/parse errors).  :-\

I do think that now having a module system in place does open the gates for more JS devs to start extending PhantomJS in userland (e.g. like NodeJS userland) but, unfortunately, most of the work PhantomJS core still needs accomplished is — so far as I can tell — on the C++ side of the fence. :(

Sincerely,
    James Greene

--

[Ariya Hidayat]

> I don't have a direct answer, but a possible explanation on the
> frustration; typical phantomjs users are frontend & backend Web
> developers. They eventually know javascript, but then often don't know
> C++ at all (like me). So contributing C++ code is a bit of hard for
> many users, that's probably why they feel frustration having to rely
> on someone else to make things happen.

Realistically of course we won't ask the user to always try to fix the
bug himself. C++, especially in a large code base, is not everyone's
cup of tea.

> So at least if we want people to contribute more code to phantomjs, we
> should lower the barrier for those people - especially C++/Qt
> beginners - to get started with contributing C++ code. Well, that's
> not an easy task for sure… but a «beginner's guide to Qt & phantomjs»
> with eg. how to setup an IDE, where to find basic information on the
> framework, how to test a patch, etc. would be a nice start

Lowering the barrier makes sense to me. I'm confident there are
potential contributors/prospects which may benefit from a "big
picture" overview of the code base.

I think I may start with something like "Architecture" wiki page and
we can dig deeper as needed (top-down).

> Also, what I would suggest is a better communication on the roadmap
> (targeted features & bug resolution), maybe with blog entries or
> regular posts on this very google group…

In the past I wrote some entries into the wiki page on roadmap.
However, it turned out that I overpromised and underdelivered and thus
I don't bother to do it. I still see the point in giving a rough idea
what to expect in the near future, possibly with a strong disclaimer
that this is just tentative and not set in stone.

> Last, even if the documentation is really better now than when hosted
> on google code, it still needs some effort to make it complete &
> accurate for users; for example, I realized yesterday than the frame
> manipulation API wasn't documented on the wiki… That certainly can
> create frustration :)

Obviously we could do better in this front.


Some of the issues you mention are fundamentally solveable, given
enough time and effort (which are usually our #1 enemy).

Thank you!

Regards,

[Ariya Hidayat]

> PhantomJS is the complicated project, especially Qt-Webkit bridge. And, as
> Nicolas mentioned, most of the users are web developers. They can do only
> one thing: use it, and report an issue. And getting started guide "How to be
> involved" could be very helpful. As a side note, I've already thought about
> - create a blog to start writing about hacking or contributing to PhantomJS.
> Currently PhantomJS little closed to users. Only those who are beginning
> hacking and has knowledge of C++ and Qt can understand what is going on
> inside of PhantomJS. When user coming to get our help, he often get the
> answer: read the doc, please.
> Of course, this is a special case. James Greene is doing an amazing job here
> - helps users. To solve it or improve, I think we need to create a wiki
> page, that explains how PhantomJS works: page context, sandbox, and other
> things.

Totally agree here. More materials explaining the guts of PhantomJS
will be always welcomed. Usually we need to be careful not to write
outdated stuff, but I think we should be able to manage it just fine.


Regards,

[James Greene]

Can we also agree upon a name for the PhantomJS primary context? I've heard/used many:
- PhantomJS outer space (POS? Hmm, maybe not.)
- PhantomJS outer context
- PhantomJS main context
- PhantomJS sandbox
- PhantomJS WebPage (while true based on implementation, this only ever confuses people more: "Do they mean WebPage? Or phantomjs.org?" Neither, of course.)
- etc.

~~James

[Vitaliy Slobodin]

Why not leave it like now - PhantomJS context?
And then we'll have:
PhantomJS context; WebPage context, etc.

James Greene wrote:

Can we also agree upon a name for the PhantomJS primary context? I've
heard/used many:
- PhantomJS outer space (POS? Hmm, maybe not.)
- PhantomJS outer context
- PhantomJS main context
- PhantomJS sandbox
- PhantomJS WebPage (while true based on implementation, this only
ever confuses people more: "Do they mean WebPage? Or phantomjs.org

<http://phantomjs.org>?" Neither, of course.)

- etc.

~~James

On Jan 18, 2013 12:24 AM, "Ariya Hidayat" <ariya....@gmail.com

<mailto:ariya....@gmail.com>> wrote:

    > PhantomJS is the complicated project, especially Qt-Webkit

    bridge. And, as< BR>     > Nicolas mentioned, most of the users are web developers. They

    can do only
    > one thing: use it, and report an issue. And getting started
    guide "How to be
    > involved" could be very helpful. As a side note, I've already
    thought about
    > - create a blog to start writing about hacking or contributing
    to PhantomJS.
    > Currently PhantomJS little closed to users. Only those who are
    beginning
    > hacking and has knowledge of C++ and Qt can understand what is
    going on
    > inside of PhantomJS. When user coming to get our help, he often
    get the
    > answer: read the doc, please.

    > Of course, this is a speci al case. James Greene is doing an

[match]

On Friday, January 18, 2013 6:21:50 AM UTC, Ariya Hidayat wrote:

> I don't have a direct answer, but a possible explanation on the
> frustration; typical phantomjs users are frontend & backend Web
> developers. They eventually know javascript, but then often don't know
> C++ at all (like me). So contributing C++ code is a bit of hard for
> many users, that's probably why they feel frustration having to rely
> on someone else to make things happen.

Realistically of course we won't ask the user to always try to fix the
bug himself. C++, especially in a large code base, is not everyone's
cup of tea.

> So at least if we want people to contribute more code to phantomjs, we
> should lower the barrier for those people - especially C++/Qt
> beginners - to get started with contributing C++ code. Well, that's
> not an easy task for sure… but a «beginner's guide to Qt & phantomjs»
> with eg. how to setup an IDE, where to find basic information on the
> framework, how to test a patch, etc. would be a nice start

Lowering the barrier makes sense to me. I'm confident there are
potential contributors/prospects which may benefit from a "big
picture" overview of the code base.

As a newbie and wannabe newtor (as Ivan put it) a "big picture" overview of the code base would definitely be useful.

Majority of questions I guess would still fall into "read the docs please" (I've been guilty of this... wouldn't it be great though if Google groups implements something like StackOverflow, so when you type a question it searches for previous content that is similar (?!)).

nasty comments on twitter etc I guess can only be handled with a polite link to a "Please contribute" page

The video James Greene linked to a couple of weeks ago was excellent too: http://www.youtube.com/watch?v=ulNSlES1Fds&sns=em