Frida Ok.ru

[Rocki Eibl]

Thisis problematic for security research, privacy analysis and debugging, and for control over your own device in general. It's not a purely theoretical problem either - protections like this attempt to directly block HTTPS inspection tools like HTTP Toolkit, which allow you to automatically intercept HTTPS from Android devices for inspection, testing & mocking, like so:

This depends on the target application(s) trusting the debugging proxy's certificate for HTTPS traffic. These HTTP interception and mocking techniques are super useful for testing and understanding most apps, but they have issues with the small set of hyper-vigilant apps that add extra protections aiming to lock down their HTTPS traffic and block this kind of inspection.

In the end, this is your Android device, and whether you're a security researcher checking for vulnerabilities, a developer trying to understand how an app uses its API, or a privacy advocate documenting what data an app is sharing, you should be able to see the messages that the apps you use transmit and receive on your own phone.

By default, when an Android app makes an HTTPS connection, it makes sure that it's talking to a trusted server by comparing the issuer of the server's certificate to Android's built-in list of trusted system certificate authorities.

99% of apps stick with that default. You can't change the system certificate authorities on normal devices, so this list is fairly reliable and secure. You can change it though on rooted devices and most emulators, so it's quite possible to intercept and inspect HTTPS traffic from these apps by using a debugging proxy for HTTPS interception in those environments.

Unfortunately however, the last 1% which don't stick with the default configuration are more complicated. These apps include their own custom certificate validation, to specify the exact HTTPS certificate issuers they're prepared to trust, instead of trusting all of the device's trusted certificate authorities. This ensures they will never trust a new certificate from a certificate authority that they don't explicitly recognize, and so won't accidentally expose HTTPS traffic to anybody other than the real server.

Certificate pinning used to be a much more popular technique, back before Android Nougat when Android's own certificate validation was more lax and users could easily be tricked into installing new trusted certificates on their devices. Nowadays this is more tightly controlled, and certificate pinning is much rarer, since (as we'll see) it's really security theater, and Google's own docs now specifically recommend against the practice:

For similar reasons, it's not popular on the web. There was a short-lived HTTP standard to support this (HTTP Public Key Pinning) but it's deprecated and support was removed from browsers, as it makes it far too easy to unexpectedly and irreparably (!) break applications for little security benefit.

That said, it's still used on Android in some corners, particularly by very high-profile apps (e.g. Twitter) and very security-sensitive apps (e.g. banking apps, like N26 or BBVA), all of whom are extremely protective over the details of how their APIs are used, and would prefer that prying eyes can't look too closely.

Frida lets you do things like logging every time an app calls a specific method, changing constants within built applications, recording how values within an application change or replacing methods to disable functionality entirely.

Frida supports Android, primarily using an on-device server that runs on rooted devices, and exposes an API via ADB so you can use Frida's CLI tools on your computer to transform apps on your phone on the fly.

This is very neat! But it's also quite intimidating if you're not familiar with low-level reverse engineering, since the internals and much of the documentation are very involved in the finer details of how applications work on each of the various target platforms.

If you don't have ADB already, you'll need to install it. To do so, you need to either install Android Studio and use the SDK manager UI there, or by downloading the platform tools including ADB directly as a standalone package.

If you don't have a rooted device, you can use an emulator instead. To set one up, use the AVD (Android Virtual Device) manager either from the Android Studio UI, or by running avdmanager in the standalone command-line tools.

Your emulator can use any Android version (although a recent version matching your machine's architecture is a good idea), but must use a non-'Google Play' build. Either Vanilla or 'Google APIs' is fine, but Google Play builds include restrictions similar to physical devices that limit debug access.

The last command will start Frida, and keep running silently. If it prints any output then something is probably wrong - most likely you've downloaded the server for the wrong architecture or you're not running these commands as root.

You can test this by running frida-ps -U. This will connect to the server via USB (-U) and list the details over every running process on the target device. If this shows you a list of processes, you're all good!

If you want to test this, but you're not sure what to un-pin, I've published a demo certificate pinning app at httptoolkit/android-ssl-pinning-demo. You can download a built APK of that app from its GitHub releases page and install it with adb install ./ssl-pinning-demo.apk. The package id is tech.httptoolkit.pinning_demo.

Once we have a target app, we need a script, which will rewrite the application. Frida scripts are simple JavaScript which can use Frida's API to define replacements for methods in the target application. By doing so, they can make a certificate-checking method do nothing, make a class ignore certificate pinning configuration, or almost anything else.

Writing these scripts is quite complicated. There's many small individual scripts available, designed to remove pinning from specific target apps or certain HTTPS libraries, but not many that try to remove pinning for all HTTPS traffic.

These scripts actually fully implement everything required for HTTPS interception (including proxy configuration and system certificate installation) but you can also use them independently for unpinning by itself. The script for Android certificate unpinning specifically is in the android directory as android-certificate-unpinning.js. You'll also need to include the configuration in config.js.

This script draws approaches and tricks from a wide range other public unpinning scripts, it's been tested against a huge variety of different targets already, and covers the vast majority of cases you'll find (and contributions to extend it to cover any new libraries or techniques that aren't currently covered are very welcome!). You can also go further, and use the android-certificate-unpinning-fallback.js script, which includes experimental auto-patching for obfuscated and unusual approaches that can't be covered with static rules.

If you'd like to know more about what's detected and unpinned, you can set the DEBUG_MODE variable in config.js, and you'll see output showing every detected script and whether it was patched, along with logs each time a hooked method is used.

Let's try taking a look at their HTTPS traffic. You'll need an HTTPS debugging proxy to test this - HTTP Toolkit will work, or you can use another HTTPS-intercepting proxy like Burp or Charles if you'd prefer.

Once you're intercepting the device, try opening any of the above apps and you'll see TLS connection errors in the debugging tool, and odd behaviour in the app. For example, when opening Twitter, HTTP Toolkit shows me this:

That means that an HTTP client (the Twitter app) is connecting and then rejecting the certificate immediately, without sending any requests. Each of these apps provides some errors like this, for the specific hosts whose certificates have been pinned within the app.

That said, this all depends on whether the script you use is aware of the specific certificate pinning code or APIs that are used. Whether this technique works depends entirely on the combination of target app and the Frida script.

The above script does remove certificate pinning from every built-in API or widely used library I'm aware of, and I've tested it successfully against the apps listed here and a long list of others. It's a good general-purpose script for most cases, but it won't work in absolutely 100% of certificate-pinned apps today. If you do find cases that aren't handled, I'm very interested in examples and contributions to cover more cases to help strip out as many certificate pinning implementations as possible, so do please file an issue!

Notably some apps which will go above and beyond, by implementing their own custom certificate pinning techniques from scratch, to make disabling it as difficult as possible. The prime example of this is the various Facebook apps, which all use their own custom reimplementation of TLS rather than the standard platform APIs.

It's definitely possible to automatically remove certificate pinning features from that too within the same Frida script in theory (contributions very welcome!), but it's significantly more difficult than mocking out a well-known common library, so I haven't done that yet, and so this script won't work for Facebook, Facebook Messenger, Instagram, or similar.

Fortunately that doesn't matter though, because Facebook offer a whitehat option in their apps to allow security researchers to disable certificate pinning directly, and you can just use that instead.

The next step is to start exploring further, to examine the APIs used and data leaked by other popular apps, and to help find and fix cases where this Frida script doesn't yet work, so we can stub out every last pinning API. Get testing!

3a8082e126