zerlaur sasjara elebannah

0 views
Skip to first unread message

Ling Baus

unread,
Aug 2, 2024, 12:50:27 AM8/2/24
to maipahane

This article was co-authored by wikiHow staff writer, Rain Kengly. Rain Kengly is a wikiHow Technology Writer. As a storytelling enthusiast with a penchant for technology, they hope to create long-lasting connections with readers from all around the globe. Rain graduated from San Francisco State University with a BA in Cinema.

This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources.

This article has been viewed 131,812 times.

Learn more...

Do you want Netflix to add a certain show or movie to their streaming service? Luckily, you'll be able to send suggestions to Netflix in a few simple steps. Since you can't do this through the Netflix app, you will need to use a desktop or mobile browser. You don't need a Netflix account to send suggestions, but you will need one to view Netflix shows and movies. This wikiHow will show you how to request new shows and movies from Netflix on your desktop or mobile browser.

Have you ever been dying to watch an old movie or show, but Netflix foils your plans by not offering your heart's desire on the platform? Well, thanks to Business Insider, we now know that the streaming service has a relatively under-the-radar page that allows subscribers to put in a request for titles they'd like to see become available. So perhaps all those new videos that arrive on Netflix every month aren't actually chosen at random.

Amazing, right? But there's a catch: If you were thinking that you'd request your favorite shows over and over again, don't bother. Netflix says that it keeps track of all user requests, so putting in a request for a title more than once won't do any good. But if you're lucky enough to have your pick chosen, Netflix promises it'll let you know so that you can start streaming ASAP.

And while you wait for your request to become available, might we suggest just browsing your regular ol' Netflix homepage? After all, the site does use a fancy algorithm to determine what kinds of shows and movies you like, so chances are, you'll never be without something to watch.

Late Sunday night, comedian and Brooklyn Nine-Nine star Chelsea Peretti was alerted by Netflix's Philippines account that you can request Netflix to add any TV show or movie that you can think of. This is lunacy.

A Twitter and Netflix user in the Philippines pointed out on Twitter last week that Netflix Philippines doesn't have Peretti's stand-up special, One of the Greats. On Sunday, Netflix Philippines replied with a link to its request page, which apparently is a thing.

Of course, Netflix can't just bring the most popular series or movies to its service. Once you submit a request, Netflix notes that, while they do appreciate the feedback, there are limits to what they can do:

Kellen is a science reporter at Mashable, covering space, environmentalism, sustainability, and future tech. Previously, Kellen has covered entertainment, gaming, esports, and consumer tech at Mashable. Follow him on Twitter @Kellenbeck

RHA funds free movies and TV shows from SWANK Motion Pictures. The Director of Media and Entertainment (DME) is in charge of selecting and requesting programming. Most movies and TV shows are distributed BEFORE other content providers (Netflix, HBO GO, etc.).

I'm going to talk about how we migrated a user-facing system from a synchronous request-response based system to an asynchronous one. I'm going to talk about what motivated us to embark on this journey, what system design changes we made, what were the challenges in this process, and what design choices and tradeoffs we made. Finally, I'm going to touch upon the validation process we used as we rolled out the new system.

Netflix is a streaming video service available to over 200 million members worldwide. Members watch TV shows, documentaries, and movies on many different supported devices. When they come to Netflix, they are given a variety of choices through our personalized recommendations. Users press play, sit back, and enjoy watching the movie.

While the movie plays, during the playback we collect a lot of data, for both operational and analytical use cases. Some of the data drives our product features like continue watching, which lets our members stop a movie in the middle and come back to it later to continue watching from that point on any device. The data also feeds personalization and recommendations engines, and the core business analytics.

I'm going to talk about our experience migrating one of the product features, viewing history, which lets members see their past viewing activity and optionally hide it. Let's look at our existing system before the migration. At a high level, we have the Netflix client on devices such as mobile phones, computers, laptops, and TVs, that is sending data during playback into the Netflix cloud.

First the data reaches the Gateway Service. From there it goes to the Playback API, which manages the lifecycle of the playback sessions. In addition, it sends the playback data into the Request Processor layer. Within the Request Processor, among other things, it is storing both short term and long term viewing data into persistence, which for us is Apache Cassandra, and also into a caching layer, EVCache, which lets us do really quick lookups.

Most of the time, this system worked absolutely fine. Once in a rare while, it was possible that an individual request being processed would be delayed because of a network blip, or maybe one of the Cassandra nodes slowed down for a brief time. When that happened, since this is synchronous processing, the request processing thread in the Request Processor layer had to wait. Then this in turn slowed down the upstream Playback API service, which in turn slowed down the Gateway Service itself.

Our solution to this problem was to introduce asynchronous processing into the system. Between the Playback API service and the Request Processor, we introduced a durable queue. Now when the request comes in, it's put into the durable queue, and immediately acknowledged. There is no need to wait for that request to be processed.

It turns out, Apache Kafka fits this use case pretty well. Kafka presents a log abstraction to which the producers like Playback API can append to, and then multiple consumers can then read from the Kafka logs at their own pace using offsets.

This sounds simple. But if we simply introduce Apache Kafka in between two of our processing layers, can we call it done? Not quite. Netflix operates at a scale of approximately 1 million events per second. At that scale, we encountered several challenges in asynchronous processing: data loss, processing latencies, out of order and duplicate records, and intermittent processing failures. There are also design decisions around Kafka consumer platform choice as well as cross-region aspects.

There are two potential sources of data loss. First: if the Kafka cluster itself were to be unavailable, of course, you might lose data. One simple way to address that would be to add an additional standby cluster. If the primary cluster were to be unavailable due to unforeseen reasons, then the publisher---in this case, Playback API---could then publish into this standby cluster. The consumer request processor can connect to both Kafka clusters and therefore not miss any data.

Obviously, the tradeoff here is additional cost. For a certain kind of data, this makes sense. Does all data require this? Fortunately not. We have two categories of data for playback. Critical data gets this treatment, which justifies the additional cost of a standby cluster. The other less critical data gets a normal single Kafka cluster. Since Kafka itself employs multiple strategies to improve availability, this is good enough.

Another source of data loss is at publish time. Kafka has multiple partitions to increase scalability. Each partition is served by an ensemble of servers called brokers. One of them is elected as the leader. When you are publishing into a partition, you send the data to the leader broker. You can then decide to wait for only the leader to acknowledge that the item has actually been persisted into durable storage, or you can also wait for the follower brokers to acknowledge that they have written into persistence as well. If you're dealing with critical data, it will make sense to wait for acknowledgement for all brokers of the partition. At a large scale, this has implications beyond just the cost of waiting for multiple writes.

What would happen if you were to lose the leader broker? This happened to us just a couple of months after we deployed our new architecture. If a broker becomes unavailable and you are waiting for acknowledgement from it, obviously your processing is going to slow down. That slowdown causes backpressure and unavailability, which we're trying to avoid.

If we are only waiting to get acknowledgement from just the leader broker, there's an interesting failure mode. What if you were to then lose the leader broker after a successful publish? Leader election will come up with a different leader. However, if the item that was acknowledged by the original leader was not completely replicated into the other brokers, then doing such an election of the leader could make you lose data, which is what we're trying to avoid. This is called an unclean broker leader election.

How did we handle the situation? Again, there's a tradeoff here to make. We have a producer library that is a wrapper on top of the Kafka producer client. There are two optimizations that are relevant here. First, because we use non-keyed partitions, the library is able to choose the partition it writes to. If one partition is unavailable because the leader broker is unavailable, our library can write to a different partition. Also, if the partition is on an under-replicated set of brokers---that is, the leader broker has more items than the follower leaders, and the replication has not caught up completely---then our library picks a different partition that is more well replicated.

90f70e40cf
Reply all
Reply to author
Forward
0 new messages