Can I restrict access to my site from specific domains?

[NP]

Hello all,

I'm working on a design (in python) where my users will embed a piece of javascript code on their website. This javascript code will call my application which then provides service to my users. An example of this approach is signing up for Google Analytics or Google Ads. Google gives you a piece of javascript code which you embed on your website and when users load your page, the javascript code calls Google Servers.

I'm trying to figure out what to do if a user is no longer using my service but forgets to delete the javascript code from their pages. Is their a way to prevent such sites from accessing my site without incurring a hit on my server? The option I can think of right now is to have my code catch the request and then throttle it but that still involves a hit to my server and if my client's website is being accessed by thousands of users, the hits to my server becomes considerable.

I know that Google App engine has a Denial of Service Protection but as far as I know, this works only for IP address. 

Thanks

[Anastasios Hatzis]

Hi NP,

just a few thoughts:

• I agree that ddos.yaml won't help you in your case, especially if IP addresses change consistently
• Provider per-user API keys that must be used by the javascript code for each request
• In your GAE app, the API request handlers should first check for the API key and immediately terminate with a specific HTTP status code and error message that communicate to the JS client that the AP key is temporarily over quota or terminated indefinitely; make this part as efficient as possible, so you loose as little instance runtime as possible
• Since it seems to be your own JS code: see if you can gracefully handle the Over-Quota error or the Indefinite-Terminate errors and cache the result, so the JS client will not send subsequent requests and cause unnecessary resources
• I guess, your app will also receive the original host-name that hosted the JS code, so you could store the host-name (and API key?); automatically email or contact the user that registered the host or API key to inform about troubleshooting steps
• I'm not sure if it is possible to have the JS code delivered by your app (with API keys validation), so there will be just a 404 if the API key is over-quote / terminated; but eventually cross-site scripting vulnerability would prevent this; I hope someone with more experience in JS can shed some light on this
• If you expect that too many users will not remove the JS code even after the API key has expired, you might consider if there is anything you could add to the JS code, that in this case, the users of that site will have an experience that will have the site operator be cooperative (e.g. blanking the page, or showing a huge red warning modal); that might be very dangerous though (for example if there is an error in your client code or server code or there is a temporary outage etc. you would make your users very angry or maybe you will be accountable for damages)

[NP]

Hi Anastasios,

Thanks for your response. I currently have a variation of what you suggested. I have list of 'paused' sites and the JS call to my application includes the domain name of the website. When the JS call comes in, my handler first checks if it's in the list of 'paused' sites and if so, it terminates with a 403 error. I ran some testing with a user whose page is viewed about 40,000 times a day. This basically means at least 40,000 hits to my instance. I worry that this is a big number especially if this happens with multiple sites. This is why I was wondering if there's any way to prevent access to my site.

You also suggested - 

• Since it seems to be your own JS code: see if you can gracefully handle the Over-Quota error or the Indefinite-Terminate errors and cache the result, so the JS client will not send subsequent requests and cause unnecessary resources

Can you explain the part in bold?

Thanks

NP

[Anastasios Hatzis]

Hi NP,

the idea was something like this:

• restructure your JS code and your API, so the JS client will do a single initial request (handshake?) and the server response will tell if the API key is still valid or invalid/expired
  
• the JS code stores the key's status locally (in context of the domain)
• if the API key is invalid/expired, no other instance of your JS code (including those in different browser tabs) will send another request to your API
• cache time-to-live should be reasonable (average time of a visit or so)
• in your GAE app, if a handshake request relates to an expired/invalid API key, store that key explicitly in memcache with a much longer TTL; so even if many different clients of the same site send requests, you will at least reduce your "wasted" instance hours

One more thing: If using your API service is not free, you should add something to your terms & conditions that the user has to fully pay for the API usage until the user removes your JS code and stops using your API.

I hope this was of any help :)

[Anastasios Hatzis]

One more idea, NP:

Your JS code at first page-load asks a third-party service if it's OK to use the API. For example, you host one public file for each user/site under some randomized URL unique to every site. The file contains the domain-name and API key (I guess, the API key is quite public anyway). If those do match, the JS code will know that it can send requests to your API. Again, cache the result, so every client will ask only once per visit.

If you want to exclude all clients of that site, you delete the site's file on GCS. The JS code receives a 404 and doesn't even touch your API and instance hours. You can have your app handle the files operations automatically per site / API key.

Downside is, additional time after page-load and that it adds another point of failure (e.g. outage of GCS). But for the latter case, your JS code could handle 500 errors gracefully and just send requests to the API, especially if you care more about availability of your API service than "unauthorized" usage.

Of course, you would need to compare the potential costs of both ideas.

On Wednesday, December 16, 2015 at 1:09:40 AM UTC+1, NP wrote:

[Nick (Cloud Platform Support)]

Your server might get a request via the javascript they embed from any computer in the world, and there's no way around checking the headers / request params for auth when receiving requests on the endpoint that the library targets.

Even if you provide them a special endpoint (security by obscurity here, but even in combination with actual security of a client secret) which will go down (serve 404, relatively quick and easy) when they've been "unsubscribed", this would not prevent users from triggering the now-failing API call through the JS you gave the developer to embed in their site.

The positive aspect of terminating service from your end is that the error bubbles to the third-party site's users, likely preventing them from continuing to use the service and generate API calls. In fact, terminating from your end is the only thing you can do, since you can't forcefully delete the JS you gave out, or make the clients "forget" the API endpoint, their key, etc.

If you're dealing with a malicious user who wants to repeatedly hit the failing API through site interaction, they won't get far as this is a very inefficient way to conduct a DOS, and if they start targeting you from a specific IP or a group of IPs with bad requests in more serious loads, you'll be dealing with something more on the order of DOS prevention than an auth question.

I hope this has helped clarify the boundaries a solution would have to exist within, but things phrased this simply can be deceptive. It would seem after this analysis that you had to pay for the failing requests by checking the auth in a request handler of your app, but the clever solution of Anastasios is to have Cloud Storage handle the rejection calls by the presence or absence of a given resource, avoiding instance-traffic, is quite good. Although you'd have to wonder if the added failure-point of Cloud Storage and dev-time is worth the saved few requests. As discussed above, if somebody really wanted to DOS your instances, they'd do it properly instead of using failing API requests which will not consume as much CPU and network bandwidth anyways as normal requests.

[Barry Hunter]

One way to help mitigate this would to be generate unique subdomains for each site. 

In AppEngine, can setup a wildcard * subdomain, to point to your app. So that most users just do direct to your live app. 

Then if want to stop a particular user hitting your app, can then change the DNS records. Point that specific subdomain elsewhere. THe specific DNS would means the request never even hits AppEngine. 

You could perhaps even have a second AppEngine app, that does nothing but serve a 503 or 410 or what ever error page. (or if feeling generous a empty file, which would technically valid JS) - this could perhaps be configued by the AppEngine Console. have the *.domain mapping to your production app. Then specific subdomains pointed to a differnt 'blackhole' app. 

[NP]

Thanks everybody for your suggestions. I will look into them all and try to figure out the best way forward.

From a functional perspective, I'm not currently looking at a Denial of Service attack (I guess that can only possibly come if I've become very big /successful or if it comes from a nefarious website in which case I can use the DOS.yaml file). I'm more worried about someone signing up for my service and then inexplicably violating the terms and agreements which would then lead me to 'suspending' the person's account from my side. Being that this is a new business/service, I'm trying to make sure that I don't incur any unnecessary expense. In this case, that would be bearing the cost of a suspended client's thousands of users hitting my servers.