[Voting System] Post-Hiatus Overhaul & DeepBot Updates

[Marstead]

Hey all!

I've revamped the voting spreadsheet after the return from the long hiatus. I've taken the last Survey feedback into account for this, and think it's the right path forward for the system, as it simplifies everything. In addition, I've rebuilt all of the voting commands from the ground up to fix some bugs related to the DeepBot 2018 New Year's Update, and to automate most voting system updates going forward. Here are the details:

As always, we're going to allow two more games from what used to be the two categories (Short & Long) to win the vote under the old rules before all of these changes take effect.

I'm officially calling the vote deadline for one Short (<=20 Hour) Game & one Long (>20 Hour) Game; these will be decided by Midnight PST Friday, 01/12/18. 

After this vote is decided, the new changes will take effect and outstanding hiatus SP will be disbursed. The voting spreadsheet has already mostly been updated to reflect the rules of the new system, but I've temporarily highlighted <20 Hour & >20 Hour games in green and red (hours column) so you can clearly tell which is in the lead for both categories.

Depending on which games win, I may try to fit one or both into the schedule during the upcoming Substravaganza to change things up a bit, and so we can dive directly into the new Voting System once everyone's Sub Block banks have been emptied.

Voting System Changes

• Vote Cost = Hours to Beat
• No minimum or maximum cost. If a game takes 1 hour to beat it will cost 1 SP. The theory is if a game really is that short, it will win quickly, draining SP from the system and having barely any effect on the schedule.
• Games played on Sub Block  will continue to discount the vote cost if these games are also on the spreadsheet. As there is no longer a cost maximum, there is no longer any downside to this feature.
• No Categories -- All Games Compete with Each Other
• The short vs. long category system has been dying a slow death since we deployed it back in August of 2015. The original idea was to keep specific, very long genres (namely, JRPGs) from totally dominating the schedule, but as the system has evolved and the vote commands have become more complex, we have other tools to help with that. I'm just going to make this easy and allow myself to make a fiat judgment call to defer a winning game for a cycle or two if it will make the schedule stagnant. See Deferrals, below.
• While this does mean a bunch of whale viewers can keep pumping ultra-short games to win the vote, there's a significant cost to doing this, and a minimal impact to the schedule. It should increase schedule churn while the big, long games continue to slowly climb to the top. Remember that as people keep voting up a long game toward the top of the sheet, it will get increasingly expensive to quickly pump a short game to the top.
• Two Games at a Time
• As with the old system, we'll continue to have two games running at once, and spend at least two days a week on voting system games (once the Substravaganza ends).
• If a game wins and is shorter than ~5 hours long on HLTB, we'll immediately draw the next winning game as well and schedule them back-to-back.
• Deferrals
• Most of the complexity of the old voting system stemmed from me wanting to design a perfect system that could balance the schedule and keep the channel from stagnating playing two 90-hour JRPGs at the same time. As we've thought about this together for nearly 3 years now, I really don't think there's an elegant algorithmic solution to this problem. The solution is a lot simpler -- I'll just make a judgment call to occasionally defer a winning game if I feel it will not be a good fit in the current schedule.
• If a game wins and I decide to defer it, it still won the vote, and is queued to be played at the earliest possible opportunity, when it fits best into the schedule. In the meantime, the next-place game will be drawn immediately and will be played if it fits better.
• With the extreme reduction in vote cost for short games and their advantage competing against longer titles, I don't think it's likely we'll see more than 1 deferred game in queue to be played, but if that does happen for some reason, I'll try to figure out the best and fairest way to handle it.
• Stagnant Game Removal
• Games that have been on the voting spreadsheet for a very long time without a lot of voting activity will become candidates for removal from the sheet, to open up their slots for auctions. Rather than try to determine this programmatically, I will again handle this via judgment call.
• If I feel a game has been on the sheet for too long without any real attention, I will announce that it will be removed unless it reaches a certain voting threshold within a certain time frame. I'll try to calibrate the target vote threshold based on the total votes out there.
• If the game ends up being removed, all SP spent voting for the game will be refunded to the original viewers. Any auction costs paid will also be refunded.
• Permanent Game Slots will never be affected by this.

Other than those changes, all other past voting system rules (e.g. rules for Extended playthroughs) will remain in place. 

DeepBot Voting System Rewrite

I was inspired to rewrite the entire DeepBot voting system from the ground up after a conversation in the new #behindthescenes Discord channel about DeepBot commands. Most of these changes will not affect anyone on the front end, but they will make it much easier for me to update games when a title wins in the future. (If you've followed along, errors in the voting system were plentiful, as it would take me a lot of time to add a new game into DeepBot). Importantly, I was able to fix a longstanding bug with large vote totals, and luckily discovered & fixed a new gamebreaking bug introduced in DeepBot's 2018 New Year update.

• Bug Fix: Vote Totals >999 now work correctly. Vote with confidence!
• For some reason, when DeepBot does math, it insists on using comma separators (e.g. 1000 votes becomes 1,000 votes). This breaks a ton of systems, even inside DeepBot itself.
• For example, my vote log is a *.csv file, so commas separate each piece of data I store in the log. Previously, if a game exceeded 999 votes, the comma in "1,000" would break the log file, shifting all the other data after the "1," to the right.
• If you're interested to see how I fixed this, check out the #behindthescenes channel. It was challenging!
• Bug Cleanup: Past votes for games with vote totals >999 have all been manually corrected
• This bug was preventing me from sharing vote trend data for Hollow Knight & Alien: Isolation, as both games received >1000 votes and my log was broken because of it.
• I've manually corrected every entry in the log; this issue should not present itself again.
• Bug Fix: It is now impossible to enter illegal votes. Critically, you can no longer cast NEGATIVE votes!
• While I was regression testing the voting system, I discovered something awful -- the 2018 DeepBot release made some internal change that stopped my error catching system from preventing NEGATIVE votes.
• In other words, you could type "!game01 -100" and actually reduce the vote total of !game01 by 100 votes. In addition, you'd steal 100 votes worth of SP from it!
• Fortunately, I keep strict vote logs and no one has ever cast a negative vote.
• Part of my fix also guaranteed that only integers are accepted as votes. Words, fractions, symbols, and decimals will not work. (In fact, the system will simply delete "."s, so if you try to vote 1.5 times, it will make you vote 15 times. Be careful!)
• Major Backend Improvement: All vote commands now use text file "variables" instead of having all of the game data hardcoded.
• DeepBot doesn't support custom variables, so when I designed the voting system, I wasn't able to write anything elegant like @game01Name@ for the commands to retrieve. Instead, if you saw the game's name displayed somewhere, it meant I typed it manually in about ~20 different locations. This applied to the game's vote cost, hours -- everything unique to a game required me to go the hundreds of lines of vote code and manually update. It was a very tedious, error-prone process.
• I've given up holding out for DeepBot to add custom variables. Instead, I've made my own system of custom variables using an elaborate directory of text files. It makes the commands really ugly and even harder for a human to parse (Check #behindthescenes for some examples), but it also means all of these values are now stored in easy to access, easy to update, one-change-affects-all text files.
• In the future, if I have to replace the game data for !game01, I only need to edit one text file and all of the related commands will automatically update. It'll significantly reduce opportunities for me to make a mistake--and boy did I catch a bunch of mistakes when I was converting to the new system! There should no longer be a long wait for me to get these new games added to the system and ready for voting--they should be ready within minutes of me getting the auction winner's game choice.

Upcoming Changes: Once we call the new vote, I would like to look again into finding a way to write a chat command that will display the top 5 games on the voting spreadsheet, without having to navigate to it. It turns out this is really hard to do in DeepBot--I'm basically writing a sort algorithm from scratch in a system that is not fun to code in--but it should be made less complicated by the lack of vote categories. I'll start thinking about it this week and try to have a finished command by next weekend.

*****

Thanks everyone! We might not see the fruits of these changes for awhile because of the upcoming Substravaganza, but I'm excited to see how the system behaves. I'd like this to be the Voting System's Final Form. You've all be really patient with the constant quarterly iteration on it as we've been trying to find the perfect, balanced system, but I think I've been pretty good at reasoning things out with everyone when something falls through the cracks of the system. Keeping the rules simple and offloading wonk into streamer fiat is the right way to go. Further changes will likely be procedural on my part (How do I handle weird situations, what sorts of judgment calls do I make, how are SP being awarded at the source) and I'd like to avoid further iterative system changes, unless if something is REALLY broken.

-Dylan ("Marstead")

[Marstead]

A quick follow-up post this morning after some discussions with viewers.

I've been wanting to move toward a "No length categories, no vote cost cap/floor" system since this started, but unfortunately the timing is going to be bad for several viewers who have been championing the longer games for awhile. I'd like to open with this offer:

• Anyone who feels their championed game is very adversely affected by this change can petition me to have the game removed from the spreadsheet and all spent SP refunded.
• I'll have to reach out to everyone who helped to champion the game to confirm they want to revert it, but I have all the records to be able to pull this off.
• I would still urge people with games in this scenario to consider waiting to see how the system actually functions -- given we are still drawing 2 games at a time, the new attractiveness of incredibly short games which don't really impact the schedule may be a major SP dump for people, which could make it easier for these very long games to continue their slow rise to the top. I've made this decision based on our years of game vote data so far, which suggest this won't significantly increase the time for a very long game to win (It already takes ~12-18 months from a solo champion).
• That said, I'm going to start looking for a way to leverage the existing vote discount system to help steadily-championed games that have sat on the sheet for a long time.
• People have been suggesting some variant of this for a long time -- e.g., if a game has been on the sheet for too long, reduce its cost by X. I've been hesitant to support it because it rewards games for not getting enough attention to win. However, this wouldn't apply to the sorts of games I'm covering in this post -- games that are getting supported very steadily, but aren't winning because they are so long.
• In the new paradigm, it makes a little more sense, and since I've already added rules for striking stagnant games from the sheet that look at vote activity over time, I could apply similar rules to discounting. In other words, if a game has very steady, active voting and still isn't winning because it is simply too long, it could start to accrue small retroactive discounts over time.
• I'll need to think this week on a fair system for this and run some trials to see how it would affect the sheet. If you have ideas on how to make this work in the new system, let me know, as I'd like them to be effective Friday night with the rest of the changes here.

Here is the main idea I'm considering, which I'm actually really liking as I think about its implications for the schedule:

• Longer games that reach certain SP Spent targets during their life on the voting sheet get bonus "Preview Stream" time: The existing discount system reduces the cost to vote for a game based on how many hours it has been featured on stream. To keep things elegant, we can leverage that same discount system, and award bonus "Preview" time for voting spreadsheet games that reach crazy SP Spent thresholds.
• Whenever a game reaches a certain threshold of SP spent, it is automatically featured for one day on the channel in a "preview" stream. We actually play the game for a session even though it hasn't won yet, and then reduce the game's vote cost and affect its vote total retroactively based on the # hours we play it.
• I can base the SP targets for these preview streams on how much SP was required to push a game to win the voting spreadsheet back when the game was originally added to the sheet.
  
• Because of SP inflation, the average amount of SP required for a game to win is constantly rising. This an elegant way to make things easier on games that have sat on the sheet for a long time, and to correct for the wildly changing voting system rules over the past couple years. This way, we set the bar for each game based on what it would have been competing against when it was first added to the sheet.
  
• For example, say Bob added a long game to the spreadsheet in January 2016. Back then, it only took about ~12000 SP Spent for a game of its category to win the vote (today, comparable games require 20000-30000 SP to win). Based on that #, I might set the following targets for preview streams:
• 6000 SP Spent: 4-hour preview stream unlocked
• 12000 SP Spent: 4-hour preview stream unlocked
• 18000 SP Spent: 4-hour preview stream unlocked
• 24000 SP Spent: 4-hour preview stream unlocked
• 30000 SP Spent: 4-hour preview stream unlocked
• In this system, by the time Bob has spent about as much on the game as it would take for another modern title to win the vote, it has already been played for about 20 hours on stream, which is pretty good bang for the SP buck for the person championing the game. In addition, the vote cost has now been discounted by 20.
• "Bonus Streams" will find a way to fit into the schedule.
• I'll try to slot them into Freeplay Day, New Game Day, or in the extra hours of Sub Block Days when it makes sense. If we have a lot of these bonus streams built up, they can temporarily take up one of the Voting Spreadsheet days (to maybe give us a break from the Longer game we're currently playing).
• SP Targets for Bonus Streams will be managed in a separate sheet/chart, as this system only really affects very dedicated regulars.
• The existing discounting system is already pretty easy to understand, and this just increases "bonus" time for games on the schedule. The inner workings of the system don't need to be visible on the voting spreadsheet's front page, or even really mentioned in the main rules call-out. If anything, they'll be in the FAQ.
• Bonus Streams and targets will only be made available for longer games.
• It would be disruptive to play a 10-hour game for 4-hours and then not come back to it for weeks or months, so this will only affect games above a certain length. I feel like 30+ hour games will be the least disrupted by this, so I might set the threshold there. But to keep things fair, I might also go ahead and make a judgement call for games near this margin to decide whether they qualify for bonus streams. For example, a super long 28-hour Metroidvania like La-Mulana wouldn't be a good candidate for this system, since it should already have a reasonable chance to win the vote, and Metroidvanias struggle when you wait too long to play them between sessions. But a 25-hour linear JRPG might be a better fit. 
• These Bonus Streams will mitigate the schedule stagnation effect of a very long game winning the voting spreadsheet by spreading its gameplay over a longer period of time, while simultaneously giving voters milestone rewards (hours to experience the game) on their 1-2 year journey to win the vote.
• Really long games can begin to grate on me (see Persona), largely due to their impact on the rest of the schedule. I feel like I would be able to enjoy these more in smaller, more distant doses than every week for 4-5 months.

I'm pretty excited about this idea the more I think about it, because I think it helps solve the problem where folks only feel like they see the results of their votes if they stick to incredibly short games. It diffuses the voting spreadsheet a bit instead of making it all about winner-takes-all, and addresses the schedule stagnation problem for really long games. Even if a discounted 90-hour game wins the vote, it will occasionally be interrupted by "Preview" streams for other long games, and it won't get so weary.

If everyone seems okay with this system, I'll start putting together a sheet with a list of games affected by this system, their SP spent targets, and their current SP spent. I'll have to spend this week thinking about what math seems fair for these targets--I'll probably have to make some adjustments after this system goes live and we see how it works.

Thanks!

-Dylan ("Marstead")