Ratings Abuse

[sporc...@gmail.com]

I've wanted to write about this for a while, but events today finally prompted me to do so. I can't speak for anyone else, but for a little over a month, just about all my quizzes have been subjected to ratings abuse. Countless 1 or 2-bombs, and not a single comment. If someone was so frustrated with a quiz that they felt the need to give a 1 rating, you'd think they'd want to say why. Similarly, I'll get 3s on slideshows that I spent hours working on. There aren't any technical issues. Their ratings have no justification, and possibly they're just not a fan of the image used. Who knows? They don't make any comments about it. 

I don't mean to sound paranoid, but I think people are purposefully doing it in some cases. I noticed a map quiz had terrible ratings without many plays. I wondered if something was wrong with it that I wasn't seeing. I finally decided to ask Admin about it, and they said it had received multiple 1-ratings from users who didn't even play the game. They removed the ratings, but at that point, it was long past the recent quiz board and any potential it had was gone. I understand that there's only so much that can be done. To be clear, this isn't an attack on Admin! I'm thankful they looked into it and made a correction to the rating. It is consistently happening to me though. One science quiz that I fact-checked multiple times, which was difficult, but doable, has barely above a 3.5. Just today, a sports quiz I released had 20 plays and the only ranking at that point was mine. Someone gave it a 1, so now it's sitting with a 3.00 rating. The quiz was fairly complex to make. I spent time to research and make sure the answers were right. I fact checked it, just like the map and science one (and all of them for that matter!). Still, a user or users clearly aren't following the guidelines and are trolling or abusing the ratings feature. 

When I spend time to make sure the quality, facts, and layout are good, I have hopes and expectations for the quiz. I think we all do. I know everything I release won't get published or recognized by curators or editors, but a quiz's prospects must be drastically reduced when the ratings are inaccurate due to abuse. What frustrates me most is that the guidelines are there for a reason, yet I don't think they're even remotely followed. Abusers seemingly run rampant without any consequences. The logic seems to be that if they don't like one part of the quiz, perhaps an omission of what they thought should be included, they 1-bomb it. I don't know, it's all mind boggling. 

I've typically found that upon release, within 20/30 plays I always get a crippling rating on new quizzes. I find it very frustrating and discouraging. I posted on here rather than annoy Admin, again. I can't report the number of quizzes that I think have suffered abuse because the number is too high. Has anyone else had similar issues lately? Has abuse been occurring more frequently, or am I just seeing it more because of a higher volume of output? Thanks for any help, guys. 

[Zachary Huffman]

I don't think I've been targeted much recently, but it has happened in the past. Your situation seems so severe that maybe who have a 1-bombing stalker. I don't know, but I'd definitely keep reporting it and if it is one person they could lose their right to rate quizzes.

In all honesty, as a curator, I know I am more likely to consider a highly rated quiz. I understand that this can be unfair, but it is still a great way to judge the quality of most quizzes, so a 1 bomb can affect more than just the creator, as it may discourage me from bringing attention to my subcategories by picking a quality quiz.

[Sheldon]

I really try to not let that sway me, Zachary, but I'm sure it has had some effect on my picks as well.

I think we've all been there, Tim. From what I can tell, I haven't received any 1s for a while but my current Box Office series was just targeted a few days ago. Every quiz in the series was given a 3. It seems to me like 3 is the new 1-bomb because it is damaging to any new quiz's rating but it isn't bad enough that Admin can justify preventing it.

I am also pretty sure I have one user who actually just gives me a 3 for any quiz I make no matter what. When I was tracking new quizzes of mine after release, the rating dropped and the user's score appeared in my friends scores list. I quickly went to the other new release of mine, and refreshed a few times, and sure enough, the rating dropped at the same time that user's score appeared in the list. Maybe this person does it to most quizzes and isn't just targeting me, but still, like Tim said, it's pretty discouraging.

Having said that, they are just ratings. At the end of the day, the quizzes you make are still top quality, despite what some crazy 1-3 bomber thinks. It would be nice to have the rating reflect that, especially when that has an effect on the future success of the quiz, but I don't think there's a fairer way to make the rating system without taking away people's freedom to express their opinion (however wrong it may be).

The only thing that would help with this is having a breakdown of ratings. I've said this a few times in various threads before but it won't change any ratings, it'll just show how the quiz got the rating. The design and layout was styled for the old page layout, but here's what I mean:

[Quizmaster91]

I recently received two 3 ratings on quizzes in one of my newest series as well. It killed the second quiz of those two, because it got very little plays after that. It might be abuse. But it might also be the fact that, although I thought they were very nice ideas with good execution, others thought of them as only mediocre. Don't forget that it's very hard to look at your own quizzes objectively, because you worked so hard on them. I'm not saying that your quizzes are bad (I wouldn't know, I haven't played them), but there might be a possibility that somebody genuinely thinks that they are not as great as you do. If I get a bad rating, which frequently happens, I try to ask myself first: "Was this really a good quiz idea? Could I improve something? Could it be too long/short/hard/quirky/...?" I do agree that accompanying comments with ratings would be great, but it's not a perfect world and people will often rate a quiz badly if they don't like it and move on. Nothing we, or Sporcle, can do about that.

As for 3 ratings being the new 1-bombs, I don't really buy that. I give 3 ratings as well, and never to abuse someone. Remember that a 3 rating might be given because someone thinks your quiz is just... ok. Not great, not bad, but exactly in the middle. Even though a rating of 3 can completely kill a quiz, 3/5 probably be perceived (especially by casual quiz players) as neutral. And mathematically, it is. If a 3 rating is considered bad, that leaves only 3 rating possibilities, and soon a 4 rating will be considered bad meaning that you would have to rate a quiz you like five globes, also if you merely kind of like it.

Lastly, the ratings system of Sheldon is only helpful if one bombs are actually one bombs. If, like you hypothesize, 2 and 3 ratings are also trolls abusing the ratings system, it would make it impossible to tell whether there's abuse or a bad quiz. Example: 2x1, 0x2, 0x3, 4x4, 6X5 = possible abuse, 1x1, 1x2, 2x3, 3x4, 6x5 = probably a quiz that is not well-liked across the board.

P.S. The subject of one bombs is somewhat of a throdden path here on Sporcle University. If you would like to know most peoples' opinion on the matter, you could look here or here or here or here or here or here. Ok, the last one is actually just pictures of puppies to cheer you up.

[Sheldon]

Oh yes, I forgot to mention that they could be legitimate 3s and that is fair enough. That's why I said the rating system shouldn't change because you do get the freedom to rate any quiz how you feel. And that includes giving quizzes a 1. Not all 1s are 1-bombs. In my particular case, it seems to be one person who is doing it which to me is  abuse, however, it's nowhere near important enough to waste any admin time on it. Tim's case with multiple 1s though, is, and admin dealt with that which is good.

The idea with displaying what the ratings are is merely clarification. A quiz could get three 5s and one 1, and another could get four 4s. They both have the same average rating, but one is obviously more liked than the other. It would give a better indication of how people are feeling about a quiz.

[SproutCM]

I have created a series of quizzes to help me learn the top 200 HP characters for the badge, and I've been making ones for each letter of the alphabet.

Every night when I publish one, the within two hours the quiz gets a 1 rating and no other ratings.

I'm quite annoyed, and it is very clearly the same person doing it at approximately the same time every night.

The bigger problem is that people rate quizzes when they hate them, but people rarely rate quizzes when they enjoy them.  I have seen quizzes lately with 1 or 2 positive comments about how great the quiz is, but no ratings. I get even more pissed when my friends comment on my quizzes on facebook, and tell me it is great, and yet the quiz still has zero ratings.

Now, as my SIA Puzzle Hunt has started, as each quiz has been published almost all of the early ratings are 1s.  And now after a day or so, there are just as many 5s as 1s, but every time a new great comment is added to the quiz, there is no new ratings.  So really I'm more frustrated that I take the time to rate all the quizzes I play by my friends, by my sporcle colleagues or people I don't know if I enjoy them. However, i get the feeling that most people just don't bother anymore.

Sad in Sporcleland,

SproutCM/Chris

[El_Dandy]

I run into this same problem, Sprout. Granted, I don't have the 1 bombing problem that others have (I've had maybe 5 quizzes get lowish ratings, and the only one I can think of that's been intentionally given low ratings was about same sex marriage), but tons of people reach out to me to tell me that they enjoy my quizzes and don't leave ratings, allowing lower ratings to skew things.

I speak only for myself, but I (usually) only vote under the following circumstances:

1. A quiz is notably, remarkably good (or bad) -- especially if it only has a few plays. For quizzes of this nature that I rate lowly, I try to leave a comment explaining why. I also tend to favorite quizzes I really like, though I've been using it sparingly recently due to changes in my work schedule not allowing me as much Sporcle time.

2. A quiz I've played in the quiz lab has over 50 plays but no ratings.

3. A quiz has a lower rating than I'd expect. As an example, I've 5'd each of the SIA puzzle hunt quizzes so far, because they're good quizzes and nowhere near deserving of the grief some are giving them.

I realize I should probably vote more than I do, but as for one person intentionally downvoting things -- just shake the haters off. Ultimately quality quizzes will see that adjustment back toward the mean.

[MoMosMoProblems]

To be able to see breakdowns of the ratings is one of the things at the top of my Sporcle wish list. I for one would love to know the median and mode, and not just the average.

I have to confess that I don't use the rating system that well or that thoroughly. I mostly rate new quizzes, and mostly 5 globes to try and help the ones I like best. I will give 4s, but fewer of them. I hardly ever give 3s unless its average is between 3 and 4 already. I would feel mean if my 3 would significantly bring down a new quiz's rating. Same with 2s, mostly if I come across one that merits a 2, it's from a new or newish quiz maker, so I skip the rating and leave suggestions on what I think could be improved. Now, if I happen across it later and nothing has changed, I may give it the bad rating then. 1s are less of a problem, because for me 1s are the irredeemably stupid or bad ones, the ones that either are somebody's idea of a joke, or where somebody contributed a quiz with questions but no answers, that type of thing. But again, mostly I do it for newer quizzes, and mostly with the aim of promoting good ones or helping people stay away from the horrible ones, not so much to give my opinion when it's a tepid one. I rarely rate published quizzes, unless I really love one, because I figure the fact that they are published already speaks to their quality. And I'm lazy.

[Librarysquirrel]

I've received a couple of 1's late at night.  I thought they were good ideas but nothing to the extent mentioned.  It's possible that to tie in with some of the concerns about quiz limits, that things are getting muddled a bit.  At least in sports, I am finding that fewer good quizzes are staying active night after night.  Lots of minefields with lower ratings seem to be clogging up the top spots on a regular basis.  Could this be contributing to a general decline in the way people rate quizzes?  Maybe, maybe not.  It doesn't help when half the popular quizzes have ratings of 2 or 3's instead of 4+.  I'd like to see a rise of stronger classic quizzes on the top.  Maybe some sort of a lower-attention span thing going on when the quizzes people are most exposed to is minefields with these mediocre ratings rather than the traditional quizzes that I'm used to seeing that give quizzes in general a better rep.

[Librarysquirrel]

Just looking at the Sports Popular in the last 7 days of the 15 we have 5 quizzes 4+, 5 quizzes in the 3's, 3 quizzes in the 2's, and 2 quizzes in the 1's.  Not great signs for quiz ratings in general.  Hopefully, we see some strong quizzes come up. I think EP's and CP's help.  It's been sporadic lately.  I think in general if there is better quality out there, people will be more generous and perhaps put more thought into their ratings.

[puckett86]

Sprout: If you're making the quizzes to help you learn something, you might consider making them private. That would avoid the 1-bomb issue. Plus, if you're making them for yourself (or a small group), I think they should probably be private anyways. It's really silly that someone would do that to all the quizzes though. Would be nice if they would at least comment with the reason, but it sounds more like a personal vendetta to me.

On Tuesday, September 15, 2015 at 11:46:52 PM UTC-4, SproutCM wrote:

[SproutCM]

No, you misunderstand me Puckett86.

Every time a new badge comes out on a topic where I have to memorize a large set of information I don't already know (Japanese prefectures, UK Prime ministers, Top 200 Harry Potter Characters), I come up with lots of unique quizzes that have never been done, and the creation of the quizzes (and the typing in of their answers as I make them) and subsequent playing of those quizzes helps me learn the information.

These are not throwaway private quizzes as unitopical study guides for a niche audience, they are new takes on old material I think are all worthy of publication. Many of them have gotten curator and editor picks already.

My way of learning a topic, like WWE wrestlers is to write like 30 quizzes about wrestling and learn the material.

If I were going to make a quiz that should be private, I would have made it private.

Chris/SproutCM

[sporc...@gmail.com]

I appreciate everyone taking the time to post. I hate seeing that this is a problem across the board, but I'm glad to know I'm not alone in dealing with it. I feel like my rating methodology is a bit different from those who have posted. To each his own, of course. Personally, I give quizzes good ratings if I enjoy them, if they're unique or if time has obviously been spent on them. I try to not leave bad ratings that'll seriously damage the score, and if I have to do so, I always leave a comment explaining why. 

I'm also glad to know that other users have been specifically targeted in the past. I really felt like a paranoid lunatic saying that yesterday! 

[Charging Tiger]

I've been receiving 2-ratings on some of my quizzes for a little while now. It hasn't been on every quiz, but it has been noticeable enough to seem like a pattern. It is really kind of cowardly, but what can you do about it?