How to prevent GET method calls while encode URL

[Kundan Rai]

Hi All,

I am encoding long URL to set availability in YES or NO by clicking on two different URLs, the problem is when I am passing URL to encode, Bitly calls GET method of URL while encoding and system showing updated status of recipient even prior to receive SMS by end user. Is there any way to handle this problem?

Thanks in Advance

Regards,

Kundan Rai

[Peter Herndon]

Hi Kundan,

If you are seeing a user agent string of "bitlybot" in your web server's access logs, that is indeed us. Otherwise, it's someone else crawling your links by crawling Bitlinks. We do have a bot we use to retrieve some information for each new Bitlink, but our bot respects the robots.txt standard (http://www.robotstxt.org), so if you add the following stanza to your servers' robots.txt, you will see our bot no longer accesses your links. Note that we do cache a site's robots.txt for 24 hours, so it will take a day before the change takes effect. Otherwise, you could configure your web server to examine the request's user agent, and if it sees "bitlybot", you could have the server return a 403 or similar.

The robots.txt stanza to add is:

User-agent: bitlybot

Disallow: /

Regards,

---Peter

[Kundan Rai]

Hi Peter,

Thanks for your valuable reply.

I have added robots.txt on my app root and seems problem has been solved now...great!! thanks a lot.