The Curious Case of Anonymous Comments

[Derick Rethans]

Hello!

As you might already be aware of, some of the anonymously comments
posted to IPC's talks were quite rude and inappropriate. For example:

"very brave to show that code mess and praise it with confidence like
these guys. one of the worst talks at the ipc." — http://joind.in/talk/view/2170

"Great topic, good slides. But unfortunately it was quite obvious that
the speaker joined the party last night" — http://joind.in/talk/view/2161

Or some low ratings with nonsense comments:

"Slides ?" — http://joind.in/talk/view/1063

While I have nothing really against anonymous comments, as long as
they are constructive I do think that some of them should be allowed
to be "censored" by the speakers, or at least event admins. Right now
anybody could go in and mark talks as 1 start with a crap comment,
without people actually have been at the talk. This is hugely annoying
for speakers of course.

The "speaker comment" feature helps a bit, to give some feedback to
the feedback, but the poor (non-sense) rating keeps influencing the
overall rating.

Now the question is: "how to deal with this?" I can't really offer
some proper suggestions, as I am not sure whether it's wise to allow
speakers to just ignore ratings from anonymous comments if they wish
so. It's open for self-serving abuse in some way. Then again, most
anonymous comments are neither constructive or fitting, so I'd not be
against their "censoring" either. Comments?

cheers,
Derick

[Bradley Holt]

Derick,

To come up with a good answer to your question I think we first need
to know who the ratings are intended to benefit. Are the ratings for
the benefit of the speaker, the event organizer, organizers of future
events, or all of the above? If the ratings are for the speaker then
let the speaker moderate them however he or she wants to. Likewise if
the ratings are for the event organizer (i.e. let the event organizer
moderate instead of the speaker). However, if the ratings are for the
benefit of organizers of future events then letting the speaker or the
current event organizer moderate them can potentially be a disservice
to these future event organizers. I think once you know who the
audience is for the ratings then a solution to the problem will be a
lot easier to identify.

Thanks,
Bradley

--
http://bradley-holt.com/

[Chris Cornutt]

Well, one thing that can be done is make the averages minus the anonymous comments. This also means dropping some of them that are valid too, unfortunately. Another option is to allow the "down voting" of comments that are harassing or just plain dumb so they can count less towards the overall total.

I agree with Bradley, though - what really matters is who the ratings are targeted at. In a perfect world, the answer would be "everyone" and there's no worries. It just doesn't work out that way, unfortunately, since everything can't just be taken at face value.

Really, the issue here is the ratings, not the comments as much. People can read comments and tell when someone's being a jerk. Do we want to propose dropping the ratings completely from anonymous comments and only allow them to contribute with text and not contribute to the vote?

-chris

--
Senior Editor
PHPDeveloper.org
ccor...@phpdeveloper.org

[Jake Smith]

Chris,

I think your idea of letting anonymous commenters leave comments, but not leave a rating would be the most ideal situation.  If the person was really that upset or excited about the presentation then they will sign up to leave a rating.  I think the whole up/down strategy can be easily be gamed and to make it efficient would require a decent amount of complexity.

If you think anonymous users should still be able to rate presentations, then I think their "weight" shouldn't be as heavy when it comes to calculating the overall rating.

Jake Smith
[t] @jakefolio
[w] www.jakefolio.com

[Stefan Koopmanschap]

Hi,

I think what might be helpful is the upcoming feature of being able to
post an anonymous comment while logged in. I would actually suggest
that, while displayed anonymously, the comment should still be linked
to the user, so that after a while you may see a pattern of certain
users doing the same thing over and over. Those users can then be
asked to give better feedback or just stop giving feedback alltogether
as it's useless to all of the groups mentioned by Bradley.

Another option would be for the anonymous ratings to not count against
the overall score of a talk. That way, people can leave anonymous
feedback (whether it's constructive or not) but at least the rating
won't count against the overall talk rating. This will ensure that a
single "stupid" comment not be able to pull down the average rating of
that talk.

Now that I'm writing this down, I would be much more in favour of the
second option than I would for the first option.

Stefan

--
Stefan Koopmanschap
http://www.stefankoopmanschap.nl/
http://www.leftontheweb.com/

[Bastian Feder]

Hoi,

I would go for the position of being able to do anonymous comments, but
decline the possibility to rate the talk. Therfore the user have to lock
in. I think this is the best way not to hydrate the rating but being
able to get valuable feedbak.
As the rating and the amount of people who did the rate is interesting
for conference organizers, they won't read each and every comment in
detail. So declining anonymous users to rate would be my choice.

just my 5 cents
Bastian

[lornajane]

I'm starting to see a pattern here, the idea that anonymous comments
would be applicable but that these users wouldn't be able to rate
talks. This is quite a big change, affecting the site where we
comment and everywhere the comments or totals are displayed, the API,
our consuming applications and so on. I support this change and I'm
adding it to the roadmap. It'll be a while before I get to it though
so if anyone has time to work on it then let us know - mail the list
and update the wiki to say you're working on it, so we don't overlap
with one another.

Thanks to Derick for raising this issue and also to everyone who
commented.

Lorna

[Stefan Koopmanschap]

For clarity, my suggestion was not to not allow them to rate (I think
anonymous ratings *can* be useful), just to not have those ratings
affect the overall rating of the talk. This is to prevent strange
anomalies where everyone rates 4 stars, yet one anonymous user rates a
1 star and the overall rating goes down to 3.

--