API Request Throtteling - Central or federated?

[Felipe Sere]

Hi guys,

I was looking into ways to protect our services from exaggerated load by "runaway clients". Keep in mind that this is for in-house services.

My idea is to basically use different kinds of information to establish the origin of the caller.

Call this the "extraction" part of my system.

Then I have to relate this call to all other calls in a certain time box. This means I have to store a history of calls per origin caller.

Call this the "history" part of my system.

Finally I have to decide wether to allow the call or not.

Call this the "decision" part of the system.

Now, there are multiple possibilites to spread these there parts of the system:

1. I could have a gateway for all API calls that acts as the extractor, history and decision part of the system.
2. I could have filters on each individual service which acts both as extractor and decision and put the history in a separate service
3. I could have filter which simply relays the call to a service acting as extractor, history and decision parts.

The thing I don't like about the gateway approach is that that system is handling all calls for the entire landscape. Yes, it does much less than the other services, but the infrastructure it runs on is comparable and hence the load might kill it and all the services behind it won't be reachable.

The thing I don't like about any filter approach is that the service still has to handle the load of the call. This protects resources behind the service, but the infrastructure of the actual service still gets pounded.

Are there any other options? Are there ways to mitigate the impact on any of the above options?

After deciding on the rough architecture the next step will be building the actual API :) thats gonna be more fun!

Cheers,

Felipe

[sune jakobsson]

Hi.

You are not really saying anything about the patterns on access, but if reads outnumber writes, you should looking into a HTTP cache, like https://www.varnish-cache.org/ that can be helpful in lots of ways.

Sune

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Felipe Sere]

Yes, I did not mention any requests patterns. 

Assume that 70% of the time a vanilla http cache will not be sufficient because the request is either a write and/or has an authorization header (which can be used for the extraction part).

--
Gruß,
Felipe

[Amir]

Hi,

I would go for a standalone gateway (i.e. a separate process that can be executed on another box if required) and use a fast in-memory datastore (such as redis) to gather the statistics and make the decision.

To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+unsubscribe@googlegroups.com.

Visit this group at http://groups.google.com/group/api-craft?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "API Craft" group.

To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+unsubscribe@googlegroups.com.

Visit this group at http://groups.google.com/group/api-craft?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Gruß,
Felipe

[Kevin Swiber]

Hey Felipe,

A Leaky Bucket[1] algorithm can handle rate limiting pretty effectively.  There are a few configuration options that would help with the "decision" part of your system.  I wrote an implementation in JavaScript that might be easier to grok than a wall of Wikipedia text[2].  Nginx also implements the same algorithm in its rate limiting module[3].

Per the architecture question:

The gateway approach makes a lot of sense here as it protects the machines running the important business logic pieces from being overwhelmed.  If you're worried about the overall capacity, scale at the gateway level.  If you can't beef up the box, just scale out by adding more and load balancing between them.  There are several options here, as well.  If you're running on AWS, you can look at Elastic Load Balancing.  If you're running on a corporate network, there may be utilities at the network level you can use (round-robin load balancers are common).  Lastly, if you control all the clients, you can round-robin domain names/IP addresses on the client itself.

By load balancing the gateway layer, you now enter the realm of distributed rate limiting.  This means your data store for the Leaky Bucket algorithm will likely be the same across nodes in your gateway.  In-memory key-value stores are pretty rad for this: Couchbase, Memcached, Redis.  Of course, if your heart belongs to SQL Server, Oracle, MySQL, or PostgreSQL… those will work just fine.  :)

As for the gateway software itself, there are plenty of options available from open source to commercial, but you could always roll your own if that makes more sense for you and your organization.

Good luck!

[1] http://en.wikipedia.org/wiki/Leaky_bucket

[2] https://github.com/argo/argo-rate-limiter

[3] http://wiki.nginx.org/HttpLimitReqModule

-- 

Kevin Swiber

@kevinswiber

https://github.com/kevinswiber

[Brian Jones]

Take a look at products like CloudGate to perform this type of standalone gateway type policy enforcement such as request throttling or response caching. There is an on premise option as well.

[Felipe Sere]

Hey Kevin,

very interesting links!

My idea was to hijack this and experiment with Akka (and maybe Scala).

I'll see what comes out of it and report back!

Thanks for the quick feedback.

Felipe

--
You received this message because you are subscribed to the Google Groups "API Craft" group.

To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.

[Ted M. Young [@jitterted]]

Not sure if this is useful to you, but I found a blog entry about using Repose for rate limiting: http://devops.rackspace.com/rate-limiting-with-repose-the-restful-proxy-service-engine.html. I'm looking at it since I want a reverse proxy anyway and it covers authentication and logging as well as rate-limiting.

;ted