Vodafone Nck Code
Mette Florida
When we started, the challenge was we had about 150+ developers. We have a lot more now, but to start with, we had about 150. We needed the solution to encourage ownership so that they could truly own their code end-to-end. And as a part of this, we also needed to give insight into production status at any moment in time. Finally, being an SRE team, the solution needed to follow SRE principles.
We had the service-specific dashboards, but we also needed to have monitors because we don't want people to sit there watching screens to see what's changed on the dashboard. We need it to actively call them out. As a part of creating those monitors, we also needed to place all the development teams on call so that if there was an issue they wouldn't have to look at a screen. They would get paged straight away and be ready and available to fix the issue. We went away, and we built that, and we learned a few lessons along the way.
We're using Terraform 0.11. Here on the left-hand side, we have a module or a resource to build a synthetic API test in Datadog. As you can see, the way count.index worked with Terraform 0.11 was that we would assign a count. We would look into a variable and count how many times it appeared.
I would then have a name that would then be associated with each of those, which you can see here. And it's the same thing. Vodafone UK would be called Homepage, trade-in would be called Trade-In, and the URL register-your-interest would be Register Your Interest.
This meant all the history that I have on that synthetic monitor is looking at something else. It meant that we could no longer trust what we were doing. This was quite a big cause of toil for us before Terraform 0.12 came out.
We had Terraform calling a Python script. That Python script would go into Azure DevOps. I mentioned earlier that we had our CI/CD, but all of our teams, users, and the services they owned were also stored there. That will make an API call to Azure DevOps and pull out all that information.
Next, the Python script would also go to AWS and pull all of our running Amazon ECS tasks, so then we had an indication of what was running in that given environment. Next, all that data would be input into Terraform. It would use the data from Azure DevOps, with the teams, users, and services to provision teams, users, and escalation policies within PagerDuty. So straight away, it gave us a vehicle to put all our developers on-call, but in a completely automated fashion with nothing being done manually.
It was a blessing in disguise that we did that because it meant that we had to sit with the solution calling us out for a couple of weeks. And we found out our state file had become huge. We were provisioning over 150 developers, 100 services, monitors, dashboards, different kinds of performance metrics that we were doing. It would take us 17 minutes to run Terraform.
That wasn't ideal because it could significantly delay the rates to which we could deliver. It ties in quite nicely to lesson three, which is to split your state. We split our state file into the PagerDuty users, dashboards, API tests, and monitors. This meant Terraform could run much faster, and we'd also increase the rate at which we could deliver and wouldn't be a bottleneck. That was how we provisioned total visibility across our entire estate. However, that just was calling us out.
You can see a little snippet there on the right-hand side, but all we're doing there is creating a synthetic, a duration monitor, and a PagerDuty schedule, all through code. Everything you see being declared is just the variables because the modules are already built, and all they're just pulling down those modules from our S3 bucket, running Terraform, and it's inputting those variables.
This became very valuable when we started doing destroy-and-deploy environments. If you wanted a new performance environment, you could spin it up. But we would validate the health by running every customer journey. Having this through Terraform meant that we could run it straight away without having to program anything manually.
We've found we now have a time-to-production of less than four hours. This is significantly reducing with every week or month that goes past. But last time we took this cut, four hours was our time-to-production.
From a business perspective, that's great, but what about the developers? We've enabled developers to have self-serve monitoring and alerting. They can develop whatever they want, whenever they want, safely, which will mean that even if monitoring and alerting was forgotten about, it would be automatically applied because we're running Terraform at every single deployment.
Developers have full ownership of their code, and they also have automated insights through the software delivery lifecycle. What does this mean? Well, let's say we hit an issue in a dev environment. If they want to, they can build a monitor to make sure they can capture that if it happens ever again.
But that monitor wouldn't have to be in the dev environment. They could also run that for the performance environment, SIT (system integration testing) environments, and also production. So straight away, they've had an issue, they've hit it, but they can be made aware of it if it happens anywhere else throughout the development lifecycle.
That took 17 minutes. And all that was, was just running Terraform across all of the different services to implement that. Next, with SREs, we've also got automated SLOs. SRE is all about our SLOs and SLIs, but now we've got them automated. Even if we have a new service going in, we can automatically see the SLI and then configure and work with the product owners to make those SLOs.
Finally, there's a big toil reduction. This is the time that we spend doing repetitive tasks. Having monitoring and alerting defined as code means that once we've done it, we never have to do it again. Through all of this, we now have all of our infrastructure defined as code, all of our applications defined as code, and all of our observability or monitoring defined as code. And through this, I hope you can see, there's been a significant benefit. Thank you for your time.
Vodafone brings the best live music to our customers. Available to all Vodafone customers, you will get access to a range of benefits including access to selected pre-sale tickets, reserved tickets, opportunities to win VIP experiences and tickets plus invites to exclusive Vodafone only events. Head to www.vodafone.com.au/ticket to find out more.
Vodafone customers get access to pre-sale tickets for selected Live Nation events around the country. We have early access for tickets across all price categories for Vodafone customers to purchase ahead of the general public on-sale. To access this pre-sale, customers will require a unique code which will be supplied via email or by registering for the code on the Vodafone promotion page at www.vodafone.com.au/ticket. You will need to use this code on the official ticketing partner website to unlock pre-sale access. Navigate to the password box on the ticket partner website and enter your code. Customers will receive one code per account which gives you access to purchase tickets. Once the code has been used it will no longer be valid. Pre-sales are available for a limited time or unless sold out prior. Pre-sale allocations can sell out quickly so to avoid disappointment, keep your eye on the pre-sale start date and time and be ready with your ticketing account details to ensure you secure tickets and have a smooth transaction. To find out more about a tour head to the Vodafone Ticket promotion page or www.livenation.com.au. T&Cs apply.
When a new Vodafone Pre-sale is announced, Vodafone customers who are opted into marketing communications will receive an email with the tour details. You can also head to www.vodafone.com.au/ticket to find all details about the pre-sale including tour dates and locations, artist information and access to redeem your unique pre-sale code. This page will be live during the promotion period. For more information on any past tours head to www.livenation.com.au.
You may receive your pre-sale code after the pre-sale has begun. Pre-sale ticket availability will depend on the individual artist. If pre-sale tickets have been exhausted, please see the artist page for links to general release tickets for each of their shows across Australia.
Vodafone pre-sale tickets are limited and subject to availability, and occasionally tickets can sell out extremely quickly. Head to the artist event page at Live Nation for information on further on-sales, such as the General Public on sale.
If a concert is cancelled, postponed or rescheduled, email communications will be sent out to all ticket holders by the Ticket Agent and will be communicated across Live Nation channels. Please head to www.livenation.com.au or the official ticketing partner website for more information.
For queries related to tickets, please contact the ticketing agent. For any Ticketmaster related enquiries please contact Ticketmaster at www.ticketmaster.com.au/help. For any Ticketek related enquiries please contact Ticketek at www.ticketek.com.au/help. For any Moshtix related enquiries please contact Moshtix at www.tixsupport.moshtix.com.au. For any Oztix related enquiries please contact Oztix at www.oztix.com.au/customer-support.
Final tickets are available to current Vodafone customers who successfully register for a final tickets code on the relevant artist or event ticketing page. To access a final tickets sale, Vodafone customers will require a unique code which will be supplied via email and text after successfully registering on the artist or event promotion page. You will need to use this code on the official ticketing partner website to unlock final ticket access. Navigate to the password box on the ticket partner website and enter your code. Customers will receive one code per account which gives you access to purchase tickets. Once the code has been used it will no longer be valid. Final tickets are available for a limited time and allocations can sell out quickly. To avoid disappointment, keep your eyes on the final ticket sale start date and time and be ready with your ticketing account details to ensure you secure tickets and have a smooth transaction. To find out more about a tour, head to the Vodafone Ticket. T&Cs apply.
4a15465005