Broken script
Arturo.DiDonna
downloading an option chain from CBOE. For example, you could get a
whole Microsoft option chain by typing :
php optionchain MSFT
You could get the same result by typing 'MSFT' and hitting the
Download button at http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx
Since a few days the script doesn't work anymore. :-( . I suspect that
something changed on the website. I am not familiar with php and I
can't fix things myself. I was wondering if there was any kind soul
willing to help.
Thanks in advance.
Arturo.
=======================
<?php
// cookie and error log path - SET THIS BEFORE TESTING
define(TMPFILEPATH, "/mf/home/unicorn/shell/tmp");
/*
* This script gets option chain data in a comma-delimited text file
* from the Chicago Board of Options web site www.cboe.com.
*
* Example for Microsoft (MSFT) stock options:
*
* URL syntax: http://example.com/optionchain.php?ticker=MSFT
*
* Commandline: % php optionchain.php MSFT
*/
$tickersymbol = isset($_GET['ticker']) ? $_GET['ticker']
: $_SERVER['argv'][1]; // get argument from commandline if no $_GET
$ch = curl_init(); // initialize curl
curl_setopt($ch, CURLOPT_VERBOSE, true); // verbose errors
$er = fopen(TMPFILEPATH.'/curl_err.txt', 'w'); // error log file
curl_setopt($ch, CURLOPT_STDERR, $er); // log the errors
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, TMPFILEPATH.'/cboe_cookie.txt');
curl_setopt($ch, CURLOPT_REFERER,
'http://www.cboe.com/delayedQuote/QuoteTableDownload.aspx');
curl_setopt($ch, CURLOPT_URL,
'http://www.cboe.com/delayedQuote/QuoteTableDownload.aspx');
curl_setopt($ch, CURLOPT_USERAGENT,
'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)');
// "log in" to web site by accessing first page (sets cookie)
$discard = curl_exec($ch);
// now post the form. The result will come after setting more
// cookies and receiving a redirect to a different URL,
// http://www.cboe.com/delayedQuote/QuoteData.dat
// which returns data using the query data contained in a cookie.
// AFTER that we get redirected back to the original page,
// so limit redirects to 1
curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
curl_setopt($ch, CURLOPT_POST, true); // enable HTTP POST
curl_setopt($ch, CURLOPT_POSTFIELDS,
'__EVENTTARGET='
.'&__EVENTARGUMENT='
.'&__VIEWSTATE='.urlencode('dDwtODQ5MjIyNjc7Oz5rmegY+4O27l7uWcpGd4iU
+1RpAA==')
.'&ucHeader:ucCBOEHeaderLinks:ucCBOEHeaderSearch:searchtext='
.'&ucHeader:ucCBOEHeaderLinks:ucCBOEHeaderSearch:Button1=Search'
.'&ucQuoteTableDownloadCtl:txtTicker='.$tickersymbol
.'&ucQuoteTableDownloadCtl:cmdSubmit=Download');
// Get data (RETURNS NULL FROM BROWSER, WORKS FROM COMMANDLINE ??)
$content = curl_exec($ch);
// Close resources
curl_close ($ch);
fclose($er);
// display result
print "<pre>Data:\n{$content}\nEnd</pre>\n";
?>
Arturo.DiDonna
> The following 'optionchain.php' script used to work pretty well in
> downloading an option chain from CBOE. For example, you could get a
> whole Microsoft option chain by typing :
>
> php optionchain MSFT
>
> You could get the same result by typing 'MSFT' and hitting the
> // which returns data using the query data contained in a cookie.
> // AFTER that we get redirected back to the original page,
> // so limit redirects to 1
>
> curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
> curl_setopt($ch, CURLOPT_POST, true); // enable HTTP POST
> curl_setopt($ch, CURLOPT_POSTFIELDS,
> '__EVENTTARGET='
> .'&__EVENTARGUMENT='
> .'&__VIEWSTATE='.urlencode('dDwtODQ5MjIyNjc7Oz5rmegY+4O27l7uWcpGd4iU
> +1RpAA==')
> .'&ucHeader:ucCBOEHeaderLinks:ucCBOEHeaderSearch:searchtext='
> .'&ucHeader:ucCBOEHeaderLinks:ucCBOEHeaderSearch:Button1=Search'
> .'&ucQuoteTableDownloadCtl:txtTicker='.$tickersymbol
> .'&ucQuoteTableDownloadCtl:cmdSubmit=Download');
>
> // Get data (RETURNS NULL FROM BROWSER, WORKS FROM COMMANDLINE ??)
> $content = curl_exec($ch);
>
> // Close resources
> curl_close ($ch);
> fclose($er);
>
> // display result
> print "<pre>Data:\n{$content}\nEnd</pre>\n";
> ?>
Oops. I forgot to post the output I get. This is it:
php optionchain_test.php MSFT
<pre>Data:
Invalid request.
End</pre>
Jerry Stuckle
If you can't fix it yourself, I'd recommend you hire someone to find the
problem and fix it. It shouldn't take too long for someone to do it
(no, I am not advertising, and I am not available).
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstu...@attglobal.net
==================
Arturo.DiDonna
> Arturo.DiDonna wrote:
> > The following 'optionchain.php' script used to work pretty well in
> > downloading an option chain from CBOE. For example, you could get a
> > whole Microsoft option chain by typing :
>
> > php optionchain MSFT
>
> > You could get the same result by typing 'MSFT' and hitting the
> ==================
I hope I am not breaking any rule here, but I thought that this was
the right place to ask such a question. Please, note that I am not a
trader nor I belong to any high profile hedge-fund or the likes (as
you can imagine they have their own professional, real time data feed
and don't need to resort to quickly hacked scripts). I am mainly using
this script for hobbystic purposes and I just wanted to get a pointer
to what is wrong with it.
Arturo
Geoff Berrow
<arturo....@gmail.com> wrote:
>Since a few days the script doesn't work anymore. :-( . I suspect that
>something changed on the website.
Well yes, if the script has previously been working. Is there any
documentation you can refer to?
--
Geoff Berrow (Put thecat out to email)
It's only Usenet, no one dies.
My opinions, not the committee's, mine.
Simple RFDs www.4theweb.co.uk/rfdmaker
Arturo.DiDonna
> On Thu, 4 Feb 2010 13:44:12 -0800 (PST), "Arturo.DiDonna"
>
> >Since a few days the script doesn't work anymore. :-( . I suspect that
> >something changed on the website.
>
> Well yes, if the script has previously been working. Is there any
> documentation you can refer to?
> --
> Geoff Berrow (Put thecat out to email)
> It's only Usenet, no one dies.
> My opinions, not the committee's, mine.
> Simple RFDswww.4theweb.co.uk/rfdmaker
Thanks for your answer. Unfortunately, the CBOE web site doesn't
mention any specific change made recently to their web site. No
documentation is provided by them either.
Thanks anyhow.
Arturo.
Jerry Stuckle
This is the correct place to ask such a question - but this is a place
to get user-to-user help, not free consulting services. We help each
other get something to work - not provide free consulting services.
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstu...@attglobal.net
==================
Geoff Berrow
<arturo....@gmail.com> wrote:
>Thanks for your answer. Unfortunately, the CBOE web site doesn't
>mention any specific change made recently to their web site. No
>documentation is provided by them either.
Oh, I thought this was an official interface and would be documented.
Don't suppose you know who wrote the original script?
Arturo.DiDonna
> On Fri, 5 Feb 2010 01:58:45 -0800 (PST), "Arturo.DiDonna"
>
> >Thanks for your answer. Unfortunately, the CBOE web site doesn't
> >mention any specific change made recently to their web site. No
> >documentation is provided by them either.
>
> Oh, I thought this was an official interface and would be documented.
>
> Don't suppose you know who wrote the original script?
>
> --
> Geoff Berrow (Put thecat out to email)
> It's only Usenet, no one dies.
> My opinions, not the committee's, mine.
> Simple RFDswww.4theweb.co.uk/rfdmaker
Unfortunately, the email address of the author was hidden in the
message from which I got the script. May be I can search the web a
little deeper.
Arturo
"Álvaro G. Vicario"
BTW, the first line has a typo. Where it says:
define(TMPFILEPATH, "/mf/home/unicorn/shell/tmp");
if should say:
define('TMPFILEPATH', "/mf/home/unicorn/shell/tmp");
But this should not be an issue.
This message comes from curl_exec() and this function would return FALSE
is there was an error in the transmission itself. It isn't the case so
it's the remote server the one that is sending such error message. You
should check with the Chicago Board of Options and see if they changed
something in their API. However, the page it fetches has this warning:
PLEASE NOTE: IT IS STRICTLY PROHIBITED TO DOWNLOAD DELAYED QUOTE TABLE
DATA FROM THIS WEB SITE BY USING AUTO-EXTRACTION PROGRAMS/QUERIES AND/OR
SOFTWARE. CBOE WILL BLOCK IP ADDRESSES OF ALL PARTIES WHO ATTEMPT TO DO
SO. THIS DATA IS PROPERTY OF THOMSON FINANCIAL/ILX. DOWNLOADING THIS
DATA IN ANY OTHER WAY THAN BY MANUAL TICKER SYMBOL ENTRY IS STRICTLY
PROHIBITED.
It looks like you are not allowed to do this and you've been banned.
--
-- http://alvaro.es - �lvaro G. Vicario - Burgos, Spain
-- Mi sitio sobre programaci�n web: http://borrame.com
-- Mi web de humor satinado: http://www.demogracia.com
--
Arturo.DiDonna
<alvaro.NOSPAMTH...@demogracia.com.invalid> wrote:
> -- Mi sitio sobre programación web:http://borrame.com
Thanks for your answer, Alvaro. I will check with CBOE people if this
PHP script infringes their rules. Usually, I use this script for just
a few tickers during off-peak hours. I guess this is OK, but I'll
check with them. BTW, I don't think that my IP has been banned since I
can still fetch chains using my browser.
Captain Paralytic
> The following 'optionchain.php' script used to work pretty well in
> downloading an option chain from CBOE. For example, you could get a
> whole Microsoft option chain by typing :
>
> php optionchain MSFT
>
> You could get the same result by typing 'MSFT' and hitting the
> // which returns data using the query data contained in a cookie.
> // AFTER that we get redirected back to the original page,
> // so limit redirects to 1
>
> curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
> curl_setopt($ch, CURLOPT_POST, true); // enable HTTP POST
> curl_setopt($ch, CURLOPT_POSTFIELDS,
> '__EVENTTARGET='
> .'&__EVENTARGUMENT='
> .'&__VIEWSTATE='.urlencode('dDwtODQ5MjIyNjc7Oz5rmegY+4O27l7uWcpGd4iU
> +1RpAA==')
> .'&ucHeader:ucCBOEHeaderLinks:ucCBOEHeaderSearch:searchtext='
> .'&ucHeader:ucCBOEHeaderLinks:ucCBOEHeaderSearch:Button1=Search'
> .'&ucQuoteTableDownloadCtl:txtTicker='.$tickersymbol
> .'&ucQuoteTableDownloadCtl:cmdSubmit=Download');
>
> // Get data (RETURNS NULL FROM BROWSER, WORKS FROM COMMANDLINE ??)
> $content = curl_exec($ch);
>
> // Close resources
> curl_close ($ch);
> fclose($er);
>
> // display result
> print "<pre>Data:\n{$content}\nEnd</pre>\n";
> ?>
I took a look at the web page and some of the field names have
changed. Also the __VIEWSTATE value is rather different from what you
are using.
Arturo.DiDonna
Thanks for your time, Captain. You might be paralitic, but still have
a sharp eye! :-) I'lll try to fix the script according to your
suggestions.
Thanks again.
Arturo
Captain Paralytic
You may find that the __VIEWSTATE value is dynamically generated and
you may need to capture it from the returned data from the "login"
curl_exec() step which you currently discard.
Arturo.DiDonna
Thanks again. This is letting a bit too complicate for little me. May
be I should really hire someone to do the job or find alternative
sources for option chains.
Thanks anyhow for your help and patience.
Greg Russell
> The following 'optionchain.php' script used to work pretty well in
> downloading an option chain from CBOE. For example, you could get a
> whole Microsoft option chain by typing :
>
> php optionchain MSFT
>
> You could get the same result by typing 'MSFT' and hitting the Download
> button at http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx
>
> Since a few days the script doesn't work anymore. :-( . I suspect that
> something changed on the website. I am not familiar with php and I can't
> fix things myself. I was wondering if there was any kind soul willing to
> help.
What part of the following from that URI don't you understand?
davi...@yahoo.com
How much do you want to pay for the program?
David
Thomas 'PointedEars' Lahn
^^^
> How much do you want to pay for the program?
>
> David
>
> On Thursday, February 4, 2010 at 1:44:12 PM UTC-8, Arturo.DiDonna wrote:
This is a joke, right?
--
PointedEars
Zend Certified PHP Engineer <http://www.zend.com/en/yellow-pages/ZEND024953>
<https://github.com/PointedEars> | <http://PointedEars.de/wsvn>
Twitter: @PointedEars2 | Please do not cc me./Bitte keine Kopien per E-Mail.
davi...@yahoo.com
+--------+--------+--------+--------+--------+-------+------+----------+-----------------
| strike | lprice | bprice | aprice | volume | opint | iv | exchange | opraw
+--------+--------+--------+--------+--------+-------+------+----------+-----------------
| 17.5 | 17.5 | 4.7 | 5.2 | 0 | 15 | 0 | ALL | (AKRX1616L17.5)
| 17.5 | 17.5 | 0.05 | 0.2 | 0 | 21 | 0 | ALL | (AKRX1616X17.5)
| 20 | 20 | 2.45 | 2.9 | 0 | 81 | 0 | ALL | (AKRX1616L20)
| 20 | 20 | 0.2 | 0.5 | 0 | 391 | 0 | ALL | (AKRX1616X20)
| 22.5 | 22.5 | 0.75 | 1.1 | 70 | 384 | 0.42 | ALL | (AKRX1616L22.5)
| 22.5 | 22.5 | 1 | 1.45 | 1 | 228 | 0.54 | ALL | (AKRX1616X22.5)
| 25 | 25 | 0.1 | 0.35 | 2 | 301 | 0.43 | ALL | (AKRX1616L25)
| 25 | 25 | 2.7 | 3.2 | 4 | 762 | 0.56 | ALL | (AKRX1616X25)
| 27.5 | 27.5 | 0 | 0.1 | 0 | 173 | 0 | ALL | (AKRX1616L27.5)
| 27.5 | 27.5 | 4.9 | 5.8 | 0 | 60 | 0 | ALL | (AKRX1616X27.5)
| 30 | 30 | 0 | 0.1 | 0 | 1003 | 0 | ALL | (AKRX1616L30)
| 30 | 30 | 6.7 | 8.5 | 0 | 39 | 0 | ALL | (AKRX1616X30)
davi...@yahoo.com
----------+----------+--------+--------+--------+--------+--------+-------+------+----------+-----------------+
expdate | qdate | strike | lprice | bprice | aprice | volume | opint | iv | exchange | opraw |
----------+----------+--------+--------+--------+--------+--------+-------+------+----------+-----------------+
20161216 | 20161123 | 17.5 | 17.5 | 4.7 | 5.2 | 0 | 15 | 0 | ALL | (AKRX1616L17.5) |
20161216 | 20161123 | 17.5 | 17.5 | 0.05 | 0.2 | 0 | 21 | 0 | ALL | (AKRX1616X17.5) |
20161216 | 20161123 | 20 | 20 | 2.45 | 2.9 | 0 | 81 | 0 | ALL | (AKRX1616L20) |
20161216 | 20161123 | 20 | 20 | 0.2 | 0.5 | 0 | 391 | 0 | ALL | (AKRX1616X20) |
20161216 | 20161123 | 22.5 | 22.5 | 0.75 | 1.1 | 70 | 384 | 0.42 | ALL | (AKRX1616L22.5) |
20161216 | 20161123 | 22.5 | 22.5 | 1 | 1.45 | 1 | 228 | 0.54 | ALL | (AKRX1616X22.5) |
20161216 | 20161123 | 25 | 25 | 0.1 | 0.35 | 2 | 301 | 0.43 | ALL | (AKRX1616L25) |
20161216 | 20161123 | 25 | 25 | 2.7 | 3.2 | 4 | 762 | 0.56 | ALL | (AKRX1616X25) |
20161216 | 20161123 | 27.5 | 27.5 | 0 | 0.1 | 0 | 173 | 0 | ALL | (AKRX1616L27.5) |
20161216 | 20161123 | 27.5 | 27.5 | 4.9 | 5.8 | 0 | 60 | 0 | ALL | (AKRX1616X27.5) |
20161216 | 20161123 | 30 | 30 | 0 | 0.1 | 0 | 1003 | 0 | ALL | (AKRX1616L30) |
20161216 | 20161123 | 30 | 30 | 6.7 | 8.5 | 0 | 39 | 0 | ALL | (AKRX1616X30) |
20161216 | 20161123 | 35 | 35 | 0 | 0.05 | 0 | 1910 | 0 | ALL | (AKRX1616L35) |
20161216 | 20161123 | 35 | 35 | 11.6 | 13.9 | 0 | 2 | 0 | ALL | (AKRX1616X35) |
20161216 | 20161123 | 37.5 | 37.5 | 0 | 0.1 | 0 | 51 | 0 | ALL | (AKRX1616L37.5) |
Lew Pitcher
> mysql> select * from fop where symbol = 'akrx';
> ----------+----------+--------+--------+--------+--------+--------+-------+------+----------+-----------------+
> expdate | qdate | strike | lprice | bprice | aprice | volume | opint |
> iv | exchange | opraw |
It is interesting that the column "symbol" isn't part of the resultset. I
would have expected it to be included.
>
>
> On Friday, December 9, 2016 at 11:17:10 AM UTC-8, davi...@yahoo.com wrote:
>> I saved some history option data:
>>
>>
>>
>>
+--------+--------+--------+--------+--------+-------+------+----------+-----------------
>> | strike | lprice | bprice | aprice | volume | opint | iv | exchange |
>> | opraw
>> On Thursday, December 8, 2016 at 2:07:30 AM UTC-8, Thomas 'PointedEars'
>> Lahn wrote:
>> > davi...@yahoo.com wrote in comp.lang.php:
>> > ^^^
>> > > I have the c (or c++) code to download the option data from cboe web
>> > > site. How much do you want to pay for the program?
>> > >
>> > > David
>> > >
>> > > On Thursday, February 4, 2010 at 1:44:12 PM UTC-8, Arturo.DiDonna
>> > > wrote:
>> > ^^^^
>> > >> The following 'optionchain.php' script used to work pretty well in
>> > >> downloading an option chain from CBOE. […]
>> >
>> > This is a joke, right?
First, you wait 6 years to reply to a post
Second, you post with an off-topic reply
Third, you post irrelevant data in your reply to the post calling you out, and
Fourth, you post more irrelevant (and apparently manufactured) data
--
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request
usahotj...@gmail.com
I have the code to download the data and save into mysql db.
********************
cboe uses the large string which exceeds the max string size(c, c++),
so I send the http requests with two socket requests.
"__EVENTTARGET=&__EVENTARGUMENT=...
The string size is more than 41044
Christoph M. Becker
> Now, I know you are trolling.
--
Christoph M. Becker
Jerry Stuckle
doesn't know any better.
--
==================
Remove the "x" from my email address
Jerry Stuckle
==================