Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Testing and hitting incapsula instead of the site
3,384 views
Skip to first unread message
Karl Dubost
Nov 15, 2013, 1:35:50 PM11/15/13
to compatibility
OK discovered an issue in the last couple of days.
I was trying to test a web site on the CLI, when I got:
→ http -b http://www.cinepremiere.com.mx/ User-Agent:"$FANUA"
<html><head><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"></head><iframe src="/_Incapsula_Resource?CWUDNSAI=9&incident_id=144020580022521145-62540414670897321&edet=12&cinfo=42cfd066c75384e504000000" frameborder=0 width="100%" height="100%" marginheight="0px" marginwidth="0px">Request unsuccessful. Incapsula incident ID: 144020580022521145-62540414670897321</iframe></html>
hmmm so I thought more headers would solve the issue. It didn't. I then went to the site with a desktop browser and I noticed there were asking for sitecheck with digital code to type. Once this done I could access the site.
There are two cookies added:
* incap_ses_144_95491
* visid_incap_95491
Going back on the CLI I tried again, and this time, still did not get the full Web site but the challenge. I guess it needs the cookies.
→ http -v GET http://www.cinepremiere.com.mx User-Agent:"$FOSUA" Accept:"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" Accept-Language:"en-US,en;q=0.5" Accept-En
coding:"gzip, deflate" Connection:"keep-alive"
GET / HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
Connection: keep-alive
Host: www.cinepremiere.com.mx
User-Agent: Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0
HTTP/1.1 200 OK
Cache-Control: no-cache
Connection: close
Connection: close
Content-Length: 2810
Content-Type: text/html
Set-Cookie: incap_ses_144_95491=mmcIXr3m/wlREJpY9Kn/AcZnhlIAAAAAzic5+h8Hl2w1ITqlqaxR1g==; path=/; Domain=.cinepremiere.com.mx
Set-Cookie: visid_incap_95491=YMSW6qJSSiepuRRXqwJtWsZnhlIAAAAAQUIPAAAAAADSYTV0v1eKpNd+tgf3uuSy; expires=Sun, 15 Nov 2015 17:59:51 GMT; path=/; Domain=.cinepremiere.com.mx
X-Iinfo: 8-3120028-0 0NNY RT(1384540102411 3) q(0 -1 -1 -1) r(0 -1) B10(4,314,0) U1
<html>
<head>
<META NAME="robots" CONTENT="noindex,nofollow">
<script>
(function() {
var z="";var b="7472797B766172207868723B76617220743D6E6577204461746528292E67657454696D6528293B766172207374617475733D227374617274223B7661722074696D696E673D6E65772041727261792833293B77696E646F772E6F6E756E6C6F61643D66756E6374696F6E28297B74696D696E675B325D3D22723A222B286E6577204461746528292E67657454696D6528292D74293B646F63756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B69662877696E646F772E584D4C4874747052657175657374297B7868723D6E657720584D4C48747470526571756573747D656C73657B7868723D6E657720416374697665584F626A65637428224D6963726F736F66742E584D4C4854545022297D7868722E6F6E726561647973746174656368616E67653D66756E6374696F6E28297B737769746368287868722E72656164795374617465297B6361736520303A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072657175657374206E6F7420696E697469616C697A656420223B627265616B3B6361736520313A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2073657276657220636F6E6E656374696F6E2065737461626C6973686564223B627265616B3B6361736520323A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072657175657374207265636569766564223B627265616B3B6361736520333A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2070726F63657373696E672072657175657374223B627265616B3B6361736520343A7374617475733D22636F6D706C657465223B74696D696E675B315D3D22633A222B286E6577204461746528292E67657454696D6528292D74293B6966287868722E7374617475733D3D323030297B706172656E742E6C6F636174696F6E2E72656C6F616428297D627265616B7D7D3B74696D696E675B305D3D22733A222B286E6577204461746528292E67657454696D6528292D74293B7868722E6F70656E2822474554222C222F5F496E63617073756C615F5265736F757263653F535748414E45444C3D363132353530323831393733313031393030362C31333632333834373439333731343733373833322C31363130353933393137363838353835303638352C3838373230222C66616C7365293B7868722E73656E64286E756C6C297D63617463682863297B7374617475732B3D6E6577204461746528292E67657454696D6528292D742B2220696E6361705F6578633A20222B633B646F63756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B";for (var i=0;i<b.length;i+=2){z=z+parseInt(b.substring(i, i+2), 16)+",";}z = z.substring(0,z.length-1); eval(eval('String.fromCharCode('+z+')'));})();
</script></head>
<body>
<iframe style="display:none;visibility:hidden;" src="http://my.incapsula.com/public/ga/jsTest.html" id="gaIframe"></iframe>
</body></html>
Incapsula is a proxy which is checking if the UA is wellknown.
http://www.incapsula.com/
They also have an encyclopedia of bots.
http://www.botopedia.org/
--
Karl Dubost, Mozilla
http://www.la-grange.net/karl/moz
I was trying to test a web site on the CLI, when I got:
→ http -b http://www.cinepremiere.com.mx/ User-Agent:"$FANUA"
<html><head><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"></head><iframe src="/_Incapsula_Resource?CWUDNSAI=9&incident_id=144020580022521145-62540414670897321&edet=12&cinfo=42cfd066c75384e504000000" frameborder=0 width="100%" height="100%" marginheight="0px" marginwidth="0px">Request unsuccessful. Incapsula incident ID: 144020580022521145-62540414670897321</iframe></html>
hmmm so I thought more headers would solve the issue. It didn't. I then went to the site with a desktop browser and I noticed there were asking for sitecheck with digital code to type. Once this done I could access the site.
There are two cookies added:
* incap_ses_144_95491
* visid_incap_95491
Going back on the CLI I tried again, and this time, still did not get the full Web site but the challenge. I guess it needs the cookies.
→ http -v GET http://www.cinepremiere.com.mx User-Agent:"$FOSUA" Accept:"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" Accept-Language:"en-US,en;q=0.5" Accept-En
coding:"gzip, deflate" Connection:"keep-alive"
GET / HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
Connection: keep-alive
Host: www.cinepremiere.com.mx
User-Agent: Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0
HTTP/1.1 200 OK
Cache-Control: no-cache
Connection: close
Connection: close
Content-Length: 2810
Content-Type: text/html
Set-Cookie: incap_ses_144_95491=mmcIXr3m/wlREJpY9Kn/AcZnhlIAAAAAzic5+h8Hl2w1ITqlqaxR1g==; path=/; Domain=.cinepremiere.com.mx
Set-Cookie: visid_incap_95491=YMSW6qJSSiepuRRXqwJtWsZnhlIAAAAAQUIPAAAAAADSYTV0v1eKpNd+tgf3uuSy; expires=Sun, 15 Nov 2015 17:59:51 GMT; path=/; Domain=.cinepremiere.com.mx
X-Iinfo: 8-3120028-0 0NNY RT(1384540102411 3) q(0 -1 -1 -1) r(0 -1) B10(4,314,0) U1
<html>
<head>
<META NAME="robots" CONTENT="noindex,nofollow">
<script>
(function() {
var z="";var b="7472797B766172207868723B76617220743D6E6577204461746528292E67657454696D6528293B766172207374617475733D227374617274223B7661722074696D696E673D6E65772041727261792833293B77696E646F772E6F6E756E6C6F61643D66756E6374696F6E28297B74696D696E675B325D3D22723A222B286E6577204461746528292E67657454696D6528292D74293B646F63756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B69662877696E646F772E584D4C4874747052657175657374297B7868723D6E657720584D4C48747470526571756573747D656C73657B7868723D6E657720416374697665584F626A65637428224D6963726F736F66742E584D4C4854545022297D7868722E6F6E726561647973746174656368616E67653D66756E6374696F6E28297B737769746368287868722E72656164795374617465297B6361736520303A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072657175657374206E6F7420696E697469616C697A656420223B627265616B3B6361736520313A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2073657276657220636F6E6E656374696F6E2065737461626C6973686564223B627265616B3B6361736520323A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072657175657374207265636569766564223B627265616B3B6361736520333A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2070726F63657373696E672072657175657374223B627265616B3B6361736520343A7374617475733D22636F6D706C657465223B74696D696E675B315D3D22633A222B286E6577204461746528292E67657454696D6528292D74293B6966287868722E7374617475733D3D323030297B706172656E742E6C6F636174696F6E2E72656C6F616428297D627265616B7D7D3B74696D696E675B305D3D22733A222B286E6577204461746528292E67657454696D6528292D74293B7868722E6F70656E2822474554222C222F5F496E63617073756C615F5265736F757263653F535748414E45444C3D363132353530323831393733313031393030362C31333632333834373439333731343733373833322C31363130353933393137363838353835303638352C3838373230222C66616C7365293B7868722E73656E64286E756C6C297D63617463682863297B7374617475732B3D6E6577204461746528292E67657454696D6528292D742B2220696E6361705F6578633A20222B633B646F63756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B";for (var i=0;i<b.length;i+=2){z=z+parseInt(b.substring(i, i+2), 16)+",";}z = z.substring(0,z.length-1); eval(eval('String.fromCharCode('+z+')'));})();
</script></head>
<body>
<iframe style="display:none;visibility:hidden;" src="http://my.incapsula.com/public/ga/jsTest.html" id="gaIframe"></iframe>
</body></html>
Incapsula is a proxy which is checking if the UA is wellknown.
http://www.incapsula.com/
They also have an encyclopedia of bots.
http://www.botopedia.org/
--
Karl Dubost, Mozilla
http://www.la-grange.net/karl/moz
0 new messages