Scalper Fx

[Imke]

LinkedIn and 3rd parties use essential and non-essential cookies to provide, secure, analyze and improve our Services, and to show you relevant ads (including professional and job ads) on and off LinkedIn. Learn more in our Cookie Policy.

A while ago, back in December, I was telling my wife about an article I just read on how difficult was for people all over the world to find the recently released last next-gen consoles on the market, yeah unless you have been living under a rock you probably heard about the new Microsoft Xbox Series X and Sony's PlayStation 5, these were the newest video-game consoles available on the market since mid November 2020!

The article covered in detail the whole drama faced by customers trying to find their long dreamed consoles, and how the traditional retailers were unable to keep up with the demand caused by the end of the year increase, the supply lines shortage, the pandemic situation affecting the factories, the scarcity of components, and other factors on this large Venn diagram.

The vast majority of customers were left only with online shopping as their last resort in hopes of grabbing that sweet piece of tech delivered to them! Not that fast, like their traditional counterparts they were subject to the same issues with a major one added to the ever expanding Venn diagram: SCALPERS!

The scalpers were individuals working together or alone using bots programmed to harvest online stores for consoles, leaving the stores completely empty, their only goal was to buy and resell the goods on eBay or similar websites.

The market price for consoles here in NZ is $799 for the Xbox Series X and $819 for the PlayStation 5, the scalpers were reselling the good on 2x, 3x, and sometimes 4x the original price, check these eBay listings - you will loose your mind when you do realize there are people paying for these things!

When I finish telling the story to my wife, she smiled and laughed, saying I was out of luck like everybody else and I wasn't going to get one, but I would be spend more time outside enjoying summer :)

I really wanted to buy the new Xbox, and I end up taking my wife words of "encouragement" as the needed motivation to find me one, so I decided to sit down and spend a couple of hours and write a quick and easy bot to find my very own Xbox!

Using Azure Logic Apps will be the easiest ways to accomplish our goal, this tutorial is to helper non-coders, and if you know how to write code, you can easily adapt this logic in any other language, but getting back to the topic, once we are done building our bot, the diagram above will look like the picture below in Azure Logic Apps:

We are going to start by creating a new blank Logic App, and the first control you are going to the canvas is a Timer, set it interval to 10 and frequency to minutes as seen below:

The next step is to add HTTP Request components for each of the websites we want the bot to track, all requests will be added in parallel, this is a very important thing. and if you decide to opt for a traditional top-down approach, you will need to rethink the entire thing and make adjustments to fit into this new model - on the HTTP Request change method to GET and set the URI to the respective URL from the website containing the Xbox or PlayStation you are after:

Visit the website and copy the URL directly from your browser's address bar, the URL will be used for the request, and the result from the HTTP Request will be evaluated on the next step on our parser.

The parser, it is the brain of our bot, actually the brains if you we are going to check more than one website, we will add a Variables control right after each HTTP Request, the value of the variable itself will be determined according to the contents of the page parsed by simply building an expression using not(), contains(), or a combination of both, it is simple and very effective!

And how about the value? Well, this is where the fun begins, we will parse the HTTP Request for an specific expression and the resulting value will TRUE (yes I found) or FALSE (not found), and the expression it is basically what you see on the page like in the example below:

Using the browser Developer Tools (assuming your browser of choice is Microsoft Edge), we will peek "behind the curtains" and see the page elements, in the example below we can see the source for "OUT OF STOCK" by simply hovering the mouse or finding the value by performing a search using CTRL+F, and this is the goal, we want our parser to perform the same action to tell us if it found or not a value behind the page:

The expression above find a matching string "OUT OF STOCK", what we really want is to make sure the string is not being displayed on the page, we don't care if there are more than one behind the page, there could be situations where a JavaScript condition is holding the string, and sometimes the may even holds the same string in different places in the DOM (Document Object Model), in many cases conditional factors will define the final rendering of the page.

Browse to the page you want your bot to track, once the page loads, press F12 to invoke the browser Developer Tools, from there click on the element inspector or CTRL+SHIFT+C, you will be able to use your mouse to hover over the page and easily find the element you are targeting for your parsing, once you click on the page the HTML source is highlighted:

From the example above, I will be looking only look for a string "Unavailable", my parser will return TRUE if it is not found in the entire page, otherwise it will return FALSE if any matching is found:

I built all my parsers to say TRUE when the strings/elements are not present in the pages, meaning a page is saying: "product available", in the end this leaves me with a very simple code to assemble my evaluation in one step, avoiding comparing each parser individually:

The most important thing here is to make sure all the previous steps have completed before we can evaluate their results, we need to configure this step to run after all previous steps have completed their operations, to configure this, click on the menu "...", and select the option Configure run after:

Once the Condition step is fully evaluated, the bot will proceed to its final phase: to send notifications if anything available is found; or to finish the execution, sending the bot back to bed leaving it wake up again once the Timer kicks in!

If any parser returns TRUE the Condition step will follow to all the steps inside TRUE to send the notifications through SMS and Email, on the step below I'm using Twillio, it provides full SMS capabilities to send and received message all over the world, for my little bot I will stick with its limited free service, good enough for our bot!

There are several options for SMS services in Azure, and you can pick any that meets your requirements, once the configuration above is supplied, I enter the phone number and text message to be sent, I'm doing this because I want to make sure about being notified, leaving me free from checking my e-mail every minute :)

The last notification to be configured is the email, pick Office 365 Outlook control, from the list of actions, select Send an email (V2), once the control is added to the canvas, you can easily configure its options to send the message like in the example below:

For example, if a console is found at Microsoft Store, the expression above will return "Microsoft :: -nz/store/b/xboxconsoles", you can add one for each variable (parser), so when the message arrives on your inbox, you will immediately have a full report of all findings and have the links to the websites, on the screen shot below you can see a detailed example of how the expression are built:

The last step in the flow for TRUE is to put an end to the execution, optional step but a good practice, it marks the Logic App has run and completed without errors, leaving you without negative records on your Azure Log Analytics:

One thing I'm omitting from this tutorial, the ability of using Azure Cognitive Services, employing this service would make this bot smarter, I don't want to go beyond of the basics but I will simply state here that you can use Forms Recognizer to deal with the process of placing order directly from your bot, Forms Recognizer can be used to read and fill out forms employing AI.

Text Analytics and Computer Vision are very useful tools to parse pages with complex elements that involve more than text, some websites may employ captchas and other visual clues, you can easily read them with some AI.

There is a lot of room for improvements on this simple bot, first of all, there is no error handling for the HTTP Requests, if a URL is not responding or has been changed, the call will produce an error and the whole bot execution will go down to the FALSE flow.

The HTTP Request status code is checked using outputs('HTTP')['statusCode'] matching code 201 for a successful execution, anything else is an error and the parser will return FALSE keeping our logic intact for the Condition step.

Also there are no breaks, the bot will keep running and spamming you, may want to include some logic to change the Timer interval, but if you are like me, just want the bot to alert, place the order and end up its execution, just add Azure Resource Manager and select the action Invoke Resource Operation (preview):

c80f0f1006