The Netflix Prize was an open competition for the best collaborative filtering algorithm to predict user ratings for films, based on previous ratings without any other information about the users or films, i.e. without the users being identified except by numbers assigned for the contest.
The competition was held by Netflix, a video streaming service, and was open to anyone who is neither connected with Netflix (current and former employees, agents, close relatives of Netflix employees, etc.) nor a resident of certain blocked countries (such as Cuba or North Korea).[1] On September 21, 2009, the grand prize of US$1,000,000 was given to the BellKor's Pragmatic Chaos team which bested Netflix's own algorithm for predicting ratings by 10.06%.[2]
For each movie, the title and year of release are provided in a separate dataset. No information at all is provided about users. In order to protect the privacy of the customers, "some of the rating data for some customers in the training and qualifying sets have been deliberately perturbed in one or more of the following ways: deleting ratings; inserting alternative ratings and dates; and modifying rating dates."[2]
There was some controversy as to the choice of RMSE as the defining metric. It has been claimed that even as small an improvement as 1% RMSE results in a significant difference in the ranking of the "top-10" most recommended movies for a user.[6]
Prizes were based on improvement over Netflix's own algorithm, called Cinematch, or the previous year's score if a team has made improvement beyond a certain threshold. A trivial algorithm that predicts for each movie in the quiz set its average grade from the training data produces an RMSE of 1.0540. Cinematch uses "straightforward statistical linear models with a lot of data conditioning."[7]
Using only the training data, Cinematch scores an RMSE of 0.9514 on the quiz data, roughly a 10% improvement over the trivial algorithm. Cinematch has a similar performance on the test set, 0.9525. In order to win the grand prize of $1,000,000, a participating team had to improve this by another 10%, to achieve 0.8572 on the test set.[2] Such an improvement on the quiz set corresponds to an RMSE of 0.8563.
As long as no team won the grand prize, a progress prize of $50,000 was awarded every year for the best result thus far. However, in order to win this prize, an algorithm had to improve the RMSE on the quiz set by at least 1% over the previous progress prize winner (or over Cinematch, the first year). If no submission succeeded, the progress prize was not to be awarded for that year.
To win a progress or grand prize a participant had to provide source code and a description of the algorithm to the jury within one week after being contacted by them. Following verification the winner also had to provide a non-exclusive license to Netflix. Netflix would publish only the description, not the source code, of the system. (To keep their algorithm and source code secret, a team could choose not to claim a prize.) The jury also kept their predictions secret from other participants. A team could send as many attempts to predict grades as they wish. Originally submissions were limited to once a week, but the interval was quickly modified to once a day. A team's best submission so far counted as their current submission.
Once one of the teams succeeded to improve the RMSE by 10% or more, the jury would issue a last call, giving all teams 30 days to send their submissions. Only then, the team with best submission was asked for the algorithm description, source code, and non-exclusive license, and, after successful verification; declared a grand prize winner.
The contest would last until the grand prize winner was declared. Had no one received the grand prize, it would have lasted for at least five years (until October 2, 2011). After that date, the contest could have been terminated at any time at Netflix's sole discretion.
By October 15, there were three teams who had beaten Cinematch, one of them by 1.06%, enough to qualify for the annual progress prize.[9] By June 2007 over 20,000 teams had registered for the competition from over 150 countries. 2,000 teams had submitted over 13,000 prediction sets.[3]
On November 13, 2007, team KorBell (formerly BellKor) was declared the winner of the $50,000 Progress Prize with an RMSE of 0.8712 (8.43% improvement).[14] The team consisted of three researchers from AT&T Labs, Yehuda Koren, Robert Bell, and Chris Volinsky.[15] As required, they published a description of their algorithm.[16]
The 2008 Progress Prize was awarded to the team BellKor. Their submission combined with a different team, BigChaos achieved an RMSE of 0.8616 with 207 predictor sets.[17]The joint-team consisted of two researchers from Commendo Research & Consulting GmbH, Andreas Tscher and Michael Jahrer (originally team BigChaos) and three researchers from AT&T Labs, Yehuda Koren, Robert Bell, and Chris Volinsky (originally team BellKor).[18] As required, they published a description of their algorithm.[19][20]
This was the final Progress Prize because obtaining the required 1% improvement over the 2008 Progress Prize would be sufficient to qualify for the Grand Prize. The prize money was donated to the charities chosen by the winners.
On July 25, 2009 the team "The Ensemble", a merger of the teams "Grand Prize Team" and "Opera Solutions and Vandelay United," achieved a 10.09% improvement over Cinematch (a Quiz RMSE of 0.8554).[21][22]
On June 26, 2009 the team "BellKor's Pragmatic Chaos", a merger of teams "Bellkor in BigChaos" and "Pragmatic Theory", achieved a 10.05% improvement over Cinematch (a Quiz RMSE of 0.8558). The Netflix Prize competition then entered the "last call" period for the Grand Prize. In accord with the Rules, teams had thirty days, until July 26, 2009 18:42:37 UTC, to make submissions that will be considered for this Prize.[23]
The final standing of the Leaderboard at that time showed that two teams met the minimum requirements for the Grand Prize. "The Ensemble" with a 10.10% improvement over Cinematch on the Qualifying set (a Quiz RMSE of 0.8553), and "BellKor's Pragmatic Chaos" with a 10.09% improvement over Cinematch on the Qualifying set (a Quiz RMSE of 0.8554).[25][26] The Grand Prize winner was to be the one with the better performance on the Test set.
On September 18, 2009, Netflix announced team "BellKor's Pragmatic Chaos" as the prize winner (a Test RMSE of 0.8567), and the prize was awarded to the team in a ceremony on September 21, 2009.[27] "The Ensemble" team had matched BellKor's result, but since BellKor submitted their results 20 minutes earlier, the rules award the prize to BellKor.[22][28]
The joint-team "BellKor's Pragmatic Chaos" consisted of two Austrian researchers from Commendo Research & Consulting GmbH, Andreas Tscher and Michael Jahrer (originally team BigChaos), two researchers from AT&T Labs, Robert Bell, and Chris Volinsky, Yehuda Koren from Yahoo! (originally team BellKor) and two researchers from Pragmatic Theory, Martin Piotte and Martin Chabbert.[29] As required, they published a description of their algorithm.[30]
On March 12, 2010, Netflix announced that it would not pursue a second Prize competition that it had announced the previous August. The decision was in response to a lawsuit and Federal Trade Commission privacy concerns.[31]
Although the data sets were constructed to preserve customer privacy, the Prize has been criticized by privacy advocates. In 2007 two researchers from The University of Texas at Austin (Vitaly Shmatikov and Arvind Narayanan) were able to identify individual users by matching the data sets with film ratings on the Internet Movie Database.[32][33]
On December 17, 2009, four Netflix users filed a class action lawsuit against Netflix, alleging that Netflix had violated U.S. fair trade laws and the Video Privacy Protection Act by releasing the datasets.[34] There was public debate about privacy for research participants. On March 19, 2010, Netflix reached a settlement with the plaintiffs, after which they voluntarily dismissed the lawsuit.
The duo will be joined by Nicki Nicole, and Lex Borrero as judges of the competition and super-producer Tainy will serve as mentor and music producer/supervisor. The search will take place across Latin America, the United States, and Spain.
La Firma was created by Borrero, developed by Howard T. Owens and showrunner Nikki Boella, and produced by Propagate and NTERTAIN, which was founded by Borrero, Tommy Mottola, and Range Media Partners.
The world of audiovisual online markets is rapidly changing. Not long ago, it was dominated by linear television, transmitted terrestrially, through cable networks or via satellite. Recently, streaming services such as Netflix, YouTube, Amazon Prime and others have emerged as new suppliers of audiovisual content. In this quickly changing industry, competition interrelations between such different formats such as traditional TV, videos on YouTube, and streaming via Netflix are subject to controversy. In particular, doubt is cast on services such as YouTube exerting competitive pressure on services such as Netflix and traditional TV. Based upon a survey with 2920 participants, we provide an empirical analysis of consumption behavior of audiovisual contents. Using descriptive and analytical statistics, including multiple equation models, we show that there are specific areas within audiovisual content markets, where YouTube exerts considerable competitive pressure on both Netflix and classic TV, for instance, through prime time video entertainment. However, our analysis yields differentiated results as we also identify areas, where competition intensity between different service types appear to be low, for instance, through daytime and regarding the intention to shorten waiting time.
The currently relevant online services differ in terms of business models from both traditional TV and from each other. In terms of business models, advertised-financed streaming services (AVoD; e.g., YouTube) can be distinguished from paid-for (by users) streaming services (PVoD; e.g., Netflix) (Lindstdt-Dreusicke & Budzinski 2020). While it is possible that streaming services mix these models (i.e., hybrid models, such as Spotify is doing in respect to audio streaming services), at the time of our empirical analysis (winter 2018/19), the relevant suppliers in the German market were predominantly devoted to one of the two models.Footnote 1 Obviously, business models will develop and change along with the high dynamics of the markets in question. Therefore, for the purpose of our analysis, we decided to directly pick two of the most prominent online services for the consumption of audiovisual contents, namely YouTube and Netflix, and compare them to traditional TV (the latter with almost no dynamics regarding the main providers; Budzinski & Lindstdt-Dreusicke 2020). While YouTube was and remains an obvious choice in the AVoD arena, choice is not that obvious regarding PVoD-services. According to the Audience Project (2019), Netflix is taking the leading position among the most used streaming and downloading services for the US, UK, Germany, Denmark, Sweden and Norway. For the US and Germany, Netflix is followed by Amazon Prime Video, whereas in the UK the BBC iPlayer takes the second position from Amazon Prime Video. In the Scandinavian countries, by contrast, Amazon Prime Video plays a smaller role. Virtually all available market share figures assume separate markets for services, such as YouTube and PVoD streaming services (thus, ignoring YouTube when calculating shares for Netflix and co). As of October 2019, market shares according to subscription figures in Germany display Amazon Prime Video (47%) leading from Netflix (36%) with other providers clearly lacking behind (e.g., Sky with 5.9%) (Herrmann 2019). However, Amazon ties its Prime subscription to a bundle of different services (e.g., free shipping, next day delivery, music streaming, video streaming), thus, it is not clear how many people exactly use Amazons VoD offer (El Khaoudi 2018). Based on daily usage figures, Netflix is leading the market in Germany, followed by Amazon Prime Video (Herrmann 2019).
90f70e40cf