dartegyp eachen randaly
Salomon Thoj
For development / debugging, it can be helpful to send a slice of production traffic to your laptop to dig into end-to-end behaviors. Our product, Spinnaker, consists of many services and a huge variety of usages by users. Running everything locally to exercise an end-to-end use cases is not often possible without melting a laptop. Instead, being able to run a single service and route production traffic can make things pretty easy.
Thankfully, task queues in Temporal are cheap to make. If this were not the case, a feature like traffic routing would be harder or impossible to implement at scale. In our particular use cases, these route task queues are typically somewhat short-lived.
The Netflix Temporal SDK has an activity interface called TaskQueueNamer which forms the basis of this feature. Different implementations are wired up into Spring Boot based on context. We use this TaskQueueNamer everywhere that task queue names are specified. In most cases and by default behavior, a provided name input will result in an equal output.
A non-traffic routing use case for this is our LaptopTaskQueueNamer, which renames all task queues when running an application locally so that operations running on your laptop do not interfere with applications running in the test environment (all laptops connect by default to our test Temporal clusters, rather than running it locally).
Adding this Fast Property will then rename all invocations of clouddriver-cloud-operations to h...@example.com/clouddriver-cloud-operations, which is our naming convention for laptop task queues, but only when the X-SPINNAKER-USER MDC value matches someon...@example.com. We use a ContextPropagator in the Temporal SDK to pass this information into Workflow & Activity executions.
There are two TaskQueueNamer implementations for traffic routing: RoutingLaptopTaskQueueNamer and RoutingTaskQueueNamer. The first is only wired up when spring.profiles.active includes laptop, which is automatically set when running an application on your laptop. The other is the default TaskQueueNamer that is wired up in a deployed application. Both implementations pull configuration from a TrafficRouteRepository, which stores a bunch of TrafficRouteProperty objects.
This should look similar to the shape of the YAML from above: We deserialize the YAML onto this. Each property must have a strategy defined, which informs the TaskQueueNamers on how to rename the task queue. We ship with two built-in strategies: value and laptop, but an application owner could provide their own strategy if they were motivated. The config field is open-ended and used to configure the strategy. For laptop, we just need user so we know how to rename the queue following the laptop task queue naming convention:
The RoutingTaskQueueNamer (which is used in deployed applications) then uses the repository of TrafficRouteProperty objects, finding the configured strategy and attempts to find a property that matches:
@RobZienert great write-up. Longer-term I see us implementing a similar routing system natively by the service. This post could be used as an inspiration. I filed a feature request to get this tracked.
One curiosity I have with interceptors - since the logic is performed within the Workflow thread context, would resolving the task queue name still be done through an activity? I would presume so, but just verifying.
Changing a task queue name (as well as any of the ActivityOptions) is a backward-compatible change. So there is no need to use activity to look those up in an interceptor. The only caveat is that the resolver should be relatively fast to not block the workflow thread for long which is going to trip the deadlock detector.
In my graduate proseminar on media theory and methods, I spend a great deal of time getting students to think about how they can draw on their own personal experiences and interactions with media to inform their scholarship. This was a central theme in Hop on Pop: The Politics and Pleasures of Popular Culture, which I co-edited with Jane Shattuc and Tara McPherson, which urges scholars to address the "culture that sticks to your skin," (a phrase inspired by Bruce Sterling's reference in Mirrorshades to "tech that sticks to your skin.") By this, we meant culture that is part of our everyday life, culture which provokes us either positively or negatively. The goal is to move cultural studies away from a language of distanced observation and towards an engagement that is up front and personal. It doesn't mean that we want only writing from fans (though of course it's no secret that I value the kinds of perspectives which fans bring to a topic.) It could also be a perspective that is antagonistic but open about its antagonism. It means being honest about where you are writing from and using a language which reflects your personal stakes in your topic. Popular culture is defined in part by its immediacy and it is not clear that one can meaningfully understand how it works or what it does without stepping at least temporarily into the realm of the proximate and the passionate. But it is not an easy thing to combine autobiography and theory effectively. I want to have my students struggle with what it means to balance these two pulls, to learn to reconcile these different languages and genre expectations through their writing. The students tell me that this is often the most challenging assignment they confront in the course. I have been grading these papers this weekend.Today, I wanted to share with you one of the papers to emerge from this assignment, with the permission, of course, of its author -- Debora Lui, who is a first years masters student in the Comparative Media Studies Program and one of the filmmakers working on the Project nml exemplar library. I felt that this particular essay would be of interest to my regular readers.
In the midst of two extensive knee surgeries in 2003, I discovered Netflix. Pumped up on painkillers, feeling groggy and uninspired, I went online one day to check out the service. I had vaguely heard of Netflix before, but had never been motivated to join. At the time, I had just graduated from college and was too busy with my "real" life to let my usually rampant movie-watching aspirations tie me down. When I moved back home in the Fall following graduation however, I was in a totally different situation. I had just injured both of my knees (tearing both Anterior Cruciate Ligaments - an amazing feat, I assure you) and my parents convinced me to move home in order to have the surgery I required. I was unemployed and living in the suburbs; watching movies suddenly became appealing again. I received my first Netflix DVD shortly after my first knee operation.
To this day, I have still remained a loyal subscriber of the service despite the rise of stronger competitors like Blockbuster (with its coupons for free in-store rentals) and the more hip GreenCine (with its Indie movie lists and user blogs). But what was it about that particular time and situation that allowed Netflix to become such an intrinsic part of my life? The website provides a very simple, yet seemingly generic service. The basic gist of Netflix (according to the simple "instructions" listed on their website) is that you create list of DVDs you want to watch online, you wait for them to be sent to you, watch them, and then return the DVDs through the mail. It is not apparent, then, why I felt such an attachment to Netflix in particular or why the service had such an exceptional hold on me. After closer examination, however, I realized there are three aspects of Netflix that allowed it become such an integral part of my life, my constant guide and companion. First, Netflix provided me a source for continuous escapism; second, it gave me a never-failing sense of accomplishment; and third, it allowed me a platform for on-going identity construction and reconstruction.
The first rental I received was the first disc of Dennis Potter's BBC series, The Singing Detective. Day and night, I was curling up with Potter's onscreen alter-ego Philip E. Marlow. I had not realized the irony at the time, of course. It would an understatement to say that Marlow wasn't the most loveable of characters, but there were some obvious similarities between us so I identified with him. I, too, was home-bound and bed-ridden, constantly feeling as if I was unable to participate in the world. Marlow created stories in his head to help him escape, and I watched Marlow create stories in his head in order to help me escape. It was a vicious cycle. Whether it was Marlow, the cast of characters for Cowboy Bebop, or Gregory Peck's character in Spellbound (respectively, my second and third rentals), I lived vicariously through their trials and travails.
Of course I wanted to escape - I was jobless, in post-surgery pain and just wanting to forget it all. Films were the perfect outlets through which I could continuously run away. The best thing about Netflix, though, wasn't that it provided me just one avenue for fleeing, but rather a continuous stream of raw material within which I could lose myself. I enjoyed all the conveniences that were initially advertised by the company; the three-at-a-time DVD plan was perfect for me. Unlike the far inferior one or two-at-a-time plans, where I might end up with nothing on hand while waiting for the next DVD in the mail, my plan allowed me nonstop opportunities for watching. One disc could be in the player, one on deck, and one could be sent back in expectation of another. In that way, anticipation of upcoming DVDs became as important as the experience of watching a movie itself. Browsing through Netflix's 75,000+ titles eventually became almost as satisfying as watching the movies themselves.
90f70e40cf