Blocked immediately

[Christopher Head]

A while ago I threw together a Python script that used the Octopart API to download part data (for my own personal use, on the order of a few dozen parts at a time once every weeks to months). Worked fine for a long time, but suddenly, as of today, I've been ratelimited. Below is a transcript of a TCP session my script made, and this was the FIRST PART I tried to search for:

GET /api/v2/parts/search?q=1N4148WSFSCT-ND&filters=%5B%5B%22supplier%22%2C+%5B%22Digi-Key%22%5D%5D%5D HTTP/1.1
Accept-Encoding: identity
Host: octopart.com
Connection: close
User-Agent: Python-urllib/3.2

HTTP/1.1 403 Forbidden
Content-Type: text/html; charset=utf-8
Date: Fri, 05 Oct 2012 04:57:24 GMT
Content-Length: 119
Connection: Close

{"message": "Blocked by rate limiter. Unblock yourself here: /ratelimiter/unblock", "__class__": "ClientErrorResponse"}

I went to octopart.com/ratelimiter/unblock and filled out the CAPTCHA. On submitting it sent by browser back to the front page, but running my script still failed on the very first part. I've tried this from two IP addresses now (on campus and at home). I'm reasonably certain my home IP has been the same for at least a few weeks, I haven't used Octopart for months before today, and I know nobody else here uses it. Why am I ratelimited?

(Minor side note, your 403 response is served with Content-Type text/html but obviously is not; perhaps it should be application/json?)

Thanks!

[andres]

Hi Christopher,

Sorry you ran into a problem with the rate limiter. In response to a few cases of excessive usage from some anonymous users we now require an API key for certain types of API requests. You can register for an API key here:

http://octopart.com/api/register

Incidentally, you should consider switching to the BOM API (http://octopart.com/api/documentation#rest_bom) which is optimized for the type of query you are trying to do:

http://octopart.com/api/v2/bom/match?lines=[{%22sku%22:%221N4148WSFSCT-ND%22}]&pretty_print=true

Andres

[Christopher Head]

Thanks for the quick response. My application doesn't have a URL as it isn't published anywhere for download. Is the URL actually used for anything important? I will indeed look at the BOM API and see if it makes sense for this application.

Chris

[Andres Morey]

Hi Chris,

Sorry for the late reply. Currently the url is just a placeholder so you don't need to link to a live app.

Andres

[Christopher Head]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

OK, now I tried to register for an API key. Logged in, provided the
name of my program ("DigiBom"), its url
(https://svn.thecube.ca/svn/thunderbots/trunk/electronics/digibom), and
ticked the box. Got "Yikes! An Error!". Don't worry about the T-shirt
though :)

Chris

On Fri, 5 Oct 2012 17:43:32 -0400
Andres Morey <and...@octopart.com> wrote:

> Hi Chris,
>
> Sorry for the late reply. Currently the url is just a placeholder so
> you don't need to link to a live app.
>
> Andres

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEAREDAAYFAlBvynIACgkQXUF6hOTGP7eBDACgithDp8qWFwj3k8fR231BYz9V
TmEAn3YwTVgLlUlX7+DQPykkisf9Pv39
=pd5Y
-----END PGP SIGNATURE-----

[Andres Morey]

Hi Chris,

Sorry there was a bug in the form handler. The maximum length of the url is 50 and we weren't handling that in the form validator. The problem should be fixed now:

http://octopart.com/api/register

Andres

[Christopher Head]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Awesome, now I’ve registered and got my key. Thanks for all the help!

Chris

On Sat, 6 Oct 2012 10:31:39 -0400

Andres Morey <and...@octopart.com> wrote:

> Hi Chris,
>

> Sorry there was a bug in the form handler. The maximum length of the
> url is 50 and we weren't handling that in the form validator. The
> problem should be fixed now:
> http://octopart.com/api/register
>
> Andres

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEAREDAAYFAlBzTnwACgkQXUF6hOTGP7cDxgCgm2hJxPfZ0V+rzdg0Kb1qHVT8
UbgAn3YJ8EhofwJlDv//uNUPiJFNlsYl
=1raK
-----END PGP SIGNATURE-----