Cameron: Hello. So, I'm Cameron Euro and I'm a security engineer on the Okta corporate security team. I'm here today to talk a little bit about how Okta protects our Okta tenant and a little bit about how you can do that as well. I'm joined by-
Jason: I'm Jason Pazoni I'm a solutions architect from Motorola Solutions. I have been responsible for a lot of areas in addition identity and assessment management, so I've been responsible for our deployment of Okta.
Cameron: So let's go ahead and jump right into it. Just to provide some additional context, I'm not a product guy. I don't do features. I don't do roadmaps. Literally, Okta is one of the services and security controls that I have to deal with on a daily basis as a corporate security team member. I look at it the same we I look at a lot of our other services such as like firewalls or anti-virus and chances are when we do have any security incident like a report of a phishing attack or anything like that, I do a lot of the same things you guys do. I go into the system log and I start looking around and see what happened. I think that I have a very similar perspective to a customer because we are an internal customer of Okta, at Okta.
With that out of the way, this is just some things that I've kind of picked up and kind of discovered over the last year and half that I've been with the team. So, obligatory Okta slide. I'm sure you've seen this in pretty much every single presentation at this conference, but it's pretty clear that Okta, we sit in front of all of your most important applications and services.
From a security team perspective, right, you're a security engineer or a security architect, you're constantly looking to place controls in place to protect these things, right? When we do place security controls in front of things, we want them to be as efficient and as effective as possible and we want to put them places where you get a good ROI. In this case, turning each and every single one of your downstream applications into a fortress doesn't scale. It's probably also impossible because they're all unique and different and have special snowflake features and may or may not offer you the security controls that you need. When you have Okta deployed, when you put all your security controls at the Okta layer it basically kind of trickles down and protects all of the downstream applications as well, so it scales much better and I think you get much better return for your investment from a detection standpoint.
So, monitoring Okta, first up is actually to take a look at what's going on within your Okta tenant. You do this primarily using the system log. You go into the admin section of the Okta app, you take a look at what's in the system log. The system log is a very basic event viewer. It doesn't have a lot of advance features, but we're not also trying to compete with Splunk or anything like that. I'm also going to plug right here the fact that I wrote a paper, actually, on how the security team at Okta uses the system log to investigate incidents and that's also a public document so if you search for the Okta Incident Response Guide it's on Okta.com and I'll have a link to it later on in the presentation. If you've ever looked at our marketing around the system log and you're like, how would I actually use it? You're just listing features. I really go in depth in my paper about how to use it, how to understand the data that's in it, basic querying, and filtering, and stuff like that so check that out if you're interested for more.
When you're looking at the system log, the system log is full of all kinds of Okta system events, right? These events are usually associated with some type of event name. Mentally, when I'm looking at these events I group them into two groups. The first group is user activity. These are things that your Okta users are doing. They're logging in, they're changing their passwords, they're using SSO'd access applications and this is really important from a security stand point, right? You want to know what your users doing. You can also spot malicious access or malicious activity. This is also really hard for security teams to kind of leverage. A lot of it is focused on base lining and detecting deviation from that baseline, which can be very difficult to do at scale especially for very large customers or customers that have a lot of business units. It's still something that's very relevant to your security team and when they're responding to an incident the user activity is going to be very important. From a detection standpoint, you get out of it what you put into it. So user activity is one.
The second group is Okta activity. So, Okta activity is like changes to your Okta tenant or admin type stuff, so the creation of a new admin user or the modification of a network zone, or the creation of an API token. These are things that happen within Okta. If I was to build something from scratch, this is where you probably want to start because they're very low volume for the most part. They are probably only going to be initiated by highly privileged users like your super admins, and then in a more mature organization these changes will likely only occur during change windows and they'll usually have associated change request tickets or change advisory board tickets.
In the case of someone creating a new admin user or granting admin to an existing user, in our case we'll immediately go in and see if there was a ticket associated with that action, and did it occur during the approved change window? It's very easy for you as a security team if you have that kind of setup in place, to notice any sort of change to your Okta tenant. I also think that this is a good place to start because once you can ensure the integrity of the Okta tenant, and that people aren't going to come in behind you and change things after you set it up all nice, that's also really important and you probably should figure that out before you do anything else. You want to make sure that it's set up the right way and detect any sort of modification or deviation from that.
On top of that, the third reason is changes to your Okta tenant and the admin section, such as changing policies, can broaden your attack surface. If someone goes in and they change a policy that you had set it could open you up to more types of attacks, or attacks from more different vectors. Those are definitely things that you want to audit.
Here I'm also going to plug another thing that I did, so this is my two, the first being the Okta Incident Response Guide. Okta has not done a super good job documenting what their events mean. If you were to say, ask, what does it look like in the system log when a user gets locked out? I don't know. There's no real public documentation around that. I kind of figured a lot of this out through trial and error. I started up my own Okta tenant and I just did things to it and saw what events were generated and over the last year and a half I've kind of identified and settled on the top 20 - 25ish events that I tend to go to when I'm conducting security incident investigations. What does it look like when someone accesses the admin section of the Okta tenant? Oh, well, in this case it's a user.session.accessadminapp. This is something I made for myself but I was able to get it published for this talk. It's on the Okta Security Lab's GitHub account. So, github.com/Oktasecuritylabs/cheatsheets. If you want to know what are some of the events you might be regularly dealing with when you're conducting an investigation, or you're not sure where to start from building your own alerts, these are probably good ones to start with. Be sure to check that out if that's something you're interested in doing.
Finally, bringing it all together, like I said earlier, the system log viewer is not a SIM. It's not a log analysis tool. It lacks key features to do a lot of the things you're probably going to want to do like send you an email or trigger a pager duty notification if something happens, or do any sort of long term pattern based analysis so if you wanted to, say, determine the failure rate of logins for a particular IP address over a 60 minute period you have to search in the system log, export it to CSV and then do all kinds of stuff like that, or you could already have it pre populated in a system that has those features. We support a number of integrations into existing tools such as like Splunk, LogRhythm, Sumo Logic, and I'll talk a little more about this later, but we have an API that is paginated access it's your Oktatenent.com/API/logs it's that simple. It feeds it back to you in Jason form, so you can just push it directly into elastic search if you wanted. We highly recommend that you do.
What happens once you begin to detect potentially suspicious or malicious stuff? Well, then you have to get into the investigation stage. What does this look like? Well, it looks a lot like dealing any other sort of application or web app. The logs contain all the things that you would expect like user agents, IP addresses, the API or URL endpoint that was hit, Okta specific stuff like the username or user ID. These are all high value indicators you're probably going to want to grab because during a security incident you're not only going to be in Okta, you're likely going to pivot into other data sources once you've identified these IPs. If you know that a particular is a host it is conducting malicious log ins you can then pivot and look at your other apps or say your firewall or your VPN that's not connected to Okta. The other thing that also is really useful that I don't know if you guys know is in the system log, we have a leased GEO IP database. We tag every single request and every single in the event log with the approximate city/country it came from which can be really useful for spotting anomalous traffic if you're a business that only operates in California, for example. Usefulness depending.
bcf7231420