A few points to be aware of:
-Just to be sure you're aware, this API is only available on ChromeOS.
-The getPrivateKeyHandle function is just a part of some sample code, not part of the API itself. Also, there's no function named signMessage; maybe you're referring to the signDigest part of the sample code?
In the example, the getPrivateKeyHandle function would read the private key for the given certificate from some source (possibly a file bundled with the extension itself, or on the filesystem, or over the network). Then the signDigest function would take the prepared digest/hash and use the private key to generate a signature in the format documented for the onSignDigestRequested
here. Another possibility would be that the private key isn't held locally, but rather on some server which you can talk to over the network to send it the digest/hash and have it return the signature to you.
If you aren't familiar with PKCS#1, RSA, or digital signature schemes, I suggest you read the following:
Also, I humbly suggest that if you are not already pretty familiar with the above concepts, you might view that as a warning sign that you shouldn't be using this API. If you're just playing around for curiosity/personal education that's fine, but if actual important private user/business data is at stake, I'd really suggest you get help from a security software professional who understands this stuff well, as there are a lot of subtle ways to get it wrong in a way that can put the confidentiality/integrity of data in jeopardy.