Updates - opentick
Chris Swierczewski
Just thought I'd post an update on what I've done so far.
First, I had to learn how to add .pyx files to a repo---with or
without and externs. This was tricky buiness, especially since
documentation was scant. My primary reference was point #3 on:
http://www.sagemath.org/doc/html/prog/node40.html
As a side project, I'm going to write up the documentation for this
proceedure. That is, creating a new .pxy file with accompanying .pxd
file and making the appropriate additions to setup.py and the local
all.py. This took way too long than I wished to figure it out and I
don't want to wish it on any other newbie to figure it out for
themselves.
Second, I started by creating a dummy class in an attempt to simply
extern from "OTClient.h" a ctypedef struct of the class "OTClient". I
don't do anything with it except instantiate the C++ class in my
wrapper. Come up with the following error on compile:
----------------------------------------------------------
sage: Building and installing modified SAGE library files.
Installing c_lib
scons: `install' is up to date.
Updating Cython code....
Building sage/finance/opentick.cpp because it depends on sage/finance/
opentick.pyx.
python2.5 `which cython` --embed-positions --incref-local-binop -I/
Users/cswiercz/sage/devel/sage-opentick -o sage/finance/opentick.cpp
sage/finance/opentick.pyx
Error converting Pyrex file to C:
------------------------------------------------------------
...
cdef class OpenTick:
cdef OTClient otclient
^
------------------------------------------------------------
/Users/cswiercz/sage/devel/sage-opentick/sage/finance/opentick.pxd:
2:18: Syntax error in C variable declaration
sage: Error running cython.
sage: There was an error installing modified sage library code.
cswiercz@D-69-91-148-78:~/sage/devel/sage/sage/finance$
I'll post my patches shortly so you can see what I've done. Again,
this is my first time doing this. I have to say I'm doing a pretty
good job so far hanging in there. :) The ticket number is
#3573....actually I just f'ed up my overwriting the first patch with
the second one just at this moment as I was about to post online, so
I'll have to replicate all this. Just give me a moment. (damn)
When it comes to trying it our for yourself, note that the headers
libraries are external to Sage at the moment. That is, they're simply
in /usr/local/include and /usr/local/lib. As for the method I used to
install all the OT stuff on my machine, I followed the instructions on
this site:
http://www.opentick.com/forum/viewtopic.php?t=2917
I'll write up an spkg soon so I can put them in something like
SAGE_ROOT/local/include/finance/opentick.
That's pretty much it for the status report. I know it's messy but
that's what I have at the moment. Will be at the Sage Lab tomorrow
from 10:45 - 1:00 to try to resolve these issues.
--
Chris
Chris Swierczewski
fixing it. (In case it wasn't clear, I lost my patches.) So I'll try
to make those changes tonight after compilation finishes. Boo.
--
Chris
Chris Swierczewski
So I finally have my code recognizing the OTFeed library --- the C++
library containing all the opentick functionality. However, when I
preface opentick.pyx with
cdef extern from "OTClient.h":
pass
I get the following compile error:
----------------------------------------------------------
sage: Building and installing modified SAGE library files.
Installing c_lib
scons: `install' is up to date.
Updating Cython code....
site-packages//sage/finance/opentick.pyx
Building sage/finance/opentick.cpp because it depends on sage/finance/
opentick.pyx.
python2.5 `which cython` --embed-positions --incref-local-binop -I/
Users/cswiercz/sage/devel/sage-opentick -o sage/finance/opentick.cpp
sage/finance/opentick.pyx
running install
running build
running build_py
running build_ext
building 'sage.finance.opentick' extension
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-
madd -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/
include -I/Users/cswiercz/sage/local/include/csage -I/Users/cswiercz/
sage/local//include -I/Users/cswiercz/sage/local//include/csage -I/
Users/cswiercz/sage/devel//sage/sage/ext -I/Users/cswiercz/sage/local/
include/python2.5 -c sage/finance/opentick.cpp -o build/
temp.macosx-10.3-i386-2.5/sage/finance/opentick.o -w -w
cc1plus: warning: command line option "-Wstrict-prototypes" is valid
for C/ObjC but not for C++
/usr/local/include/Socket.h:63: error: ‘socklen_t’ has not been
declared
/usr/local/include/Socket.h:78: error: field ‘addr’ has incomplete
type
/usr/local/include/Threads.h:26: error: invalid function declaration
/usr/local/include/Threads.h:53: error: invalid function declaration
/usr/local/include/Threads.h:77: error: invalid function declaration
/usr/local/include/Threads.h:92: error: invalid function declaration
/usr/local/include/OTSocket.h:36: error: ‘CriticalSection’ does not
name a type
/usr/local/include/OTDataEntity.h:19: error: invalid function
declaration
/usr/local/include/OTCommands.h:103: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:117: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:131: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:144: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:157: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:170: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:182: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:190: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:202: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:210: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:219: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:227: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:239: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:247: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:264: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:272: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:334: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:342: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTCommands.h:353: error: ‘OTDataEntity’ has not
been declared
/usr/local/include/OTCommands.h:361: error: expected ‘;’ before ‘*’
token
/usr/local/include/OTQuote.h:17: error: invalid function declaration
/usr/local/include/OTMMQuote.h:20: error: invalid function declaration
/usr/local/include/OTTrade.h:17: error: invalid function declaration
/usr/local/include/OTError.h:17: error: invalid function declaration
/usr/local/include/OTMessage.h:16: error: invalid function declaration
/usr/local/include/OTExchange.h:18: error: invalid function
declaration
/usr/local/include/OTSymbol.h:17: error: invalid function declaration
/usr/local/include/OTOHLC.h:18: error: invalid function declaration
/usr/local/include/OTEquityInit.h:17: error: invalid function
declaration
/usr/local/include/OTBookCancel.h:17: error: invalid function
declaration
/usr/local/include/OTBookChange.h:13: error: invalid function
declaration
/usr/local/include/OTBookDelete.h:15: error: invalid function
declaration
/usr/local/include/OTBookExecute.h:17: error: invalid function
declaration
/usr/local/include/OTBookOrder.h:16: error: invalid function
declaration
/usr/local/include/OTBookPriceLevel.h:20: error: invalid function
declaration
/usr/local/include/OTBookPurge.h:15: error: invalid function
declaration
/usr/local/include/OTBookReplace.h:15: error: invalid function
declaration
/usr/local/include/OTBBO.h:16: error: invalid function declaration
/usr/local/include/OTSplit.h:17: error: invalid function declaration
/usr/local/include/OTDividend.h:17: error: invalid function
declaration
/usr/local/include/OTOptionInit.h:10: error: invalid function
declaration
/usr/local/include/OTClient.h:45: error: expected ‘;’ before ‘*’ token
/usr/local/include/OTClient.h:51: error: invalid function declaration
error: command 'gcc' failed with exit status 1
Methinks this is an issue with the actual C++ libraries and headers
than with my implementation. I mean, I did all that seemed necessary
to do in setup.py to make sure that Sage is seeing this stuff at C++
code instead of C code. I'll post a patch today. (And I'll be very
careful not to accidentally delete things!) :) If anyone wants to have
a look. Again, the instructions I followed for obtaining the opentick
code can be found here:
http://www.opentick.com/forum/viewtopic.php?t=2917
--
Chris
Glenn H Tarbox, PhD
I've decided to try an alternative route. I'm going to take the
OpenTick Java API and build a Protocol Buffer wrapper and go directly to
python on the client side.
While I'm at it, I'll do a twisted client / server implementation. I
think it'll be straightforward but we'll see...
-glenn
>
> --
> Chris
> >
--
Glenn H. Tarbox, PhD || 206-494-0819 || gl...@tarbox.org
"Don't worry about people stealing your ideas. If your ideas are any
good you'll have to ram them down peoples throats" -- Howard Aiken
Glenn H Tarbox, PhD
> On Thu, 2008-07-10 at 21:05 -0700, Chris Swierczewski wrote:
> > Regarding my mess-up above, I should upgrade to 3.0.4 anyway before
> > fixing it. (In case it wasn't clear, I lost my patches.) So I'll try
> > to make those changes tonight after compilation finishes. Boo.
>
> I've decided to try an alternative route. I'm going to take the
> OpenTick Java API and build a Protocol Buffer wrapper and go directly to
> python on the client side.
>
> While I'm at it, I'll do a twisted client / server implementation. I
> think it'll be straightforward but we'll see...
I just got feedback from some other folks with knowledge of Xasax and
OpenTick. Its beginning to look like there's trouble and OpenTick might
not be viable.
This may turn out not to be the case, but given my limited resources I'm
going to focus on IB as a data source for the time being. Since I have
working python code, I'm going to finish my foolscap front end of IbPy.
Hopefully, Brett will have luck wrapping IB's API in Protocol Buffer
which will give us additional capabilities.
-glenn
Chris Swierczewski
> I just got feedback from some other folks with knowledge of Xasax and
> OpenTick. Its beginning to look like there's trouble and OpenTick might
> not be viable.
Trouble? Could you elaborate?
--
Chris
Glenn H Tarbox, PhD
did you say elaborate? :-)
OpenTick hasn't been taking new customers or allowing existing customers
to add exchanges etc. The concept was that there's some kind of
upgrade.
Well, that was about 3 months or so ago. There's no current info on the
forums... other than the sticky post that "we'll be in touch"... it all
smelled kinda fishy.
As you know, I've been in touch with Xasax and was assured OpenTick was
all imminent... not so much I guess...
Today, two folks with other relationships with OpenTick / Xasax
indicated that, generally speaking, there doesn't appear to be anybody
home. In one case, I was queried if I knew anything because they
couldn't get any information from the Xasax side. I don't know the
whole story... but it smells like its unwinding.
Anyway, maybe this is all nothing and everything is fine.
But an upgrade is a weekend.
To make it all a bit wierder, they're still up... just not taking new
customers... I can't grok that part at all... are they having power
problems??? can't plug in new servers? can't afford to upgrade the
pipes??? too many rows in their customer database???
If you're a company providing real services for critical data,
presumably you know how to handle this kinda thing. Clearly, the free
side, OpenTick, will not be the first priority... OTOH, you can't just
drop it entirely either. and, presumably you kinda know how to do the
upgrade...
>From my perspective, the value of OpenTick was that there was a solid
company doing real data, Xasax, and there was OpenTick which just
happened to be perfect for the academic side. We could build
infrastructure that would be mostly the same for both. A big win. Of
course, Xasax gets a little exposure which is, I'm guessing, why they
did OpenTick in the first place.
But, that cuts both ways... if you really screw up your free service,
the publicity gets negative... when you don't care, well, that's
probably not a good sign... because you're probably worrying about other
stuff.
Anyway, that's the information and my analysis. Until I hear
differently, its enough for me not to want to waste time. I can get
what I need from IB. Its not great for a lot but it works and they're
likely not going away.
Chris Swierczewski
Just a little update on my progress:
I established basic communication between the OTFeed library and my
opentick.pyx source file, meaning that compiling works with the
command
cdef extern from "OTClient__LINUX.h":
ctypedef struct c_OTClient "OTClient":
pass
and and empty class
cdef class Opentick:
def __cinit__(self):
print "foo"
def __dealloc__(self):
print "foo"
with nothing in the corresponding opentick.pxd file except "cdef class
Opentick:". Now the issue is as follows: I followed the two online
examples on the Sage/Cython wikis on how to wrap C++ classes.
Following them, I encountered the following familiar compile error:
Error converting Pyrex file to C:
------------------------------------------------------------
...
double exp(double)
cdef class Opentick:
cdef OTClient *otclient
^
------------------------------------------------------------
/Users/cswiercz/sage/devel/sage-opentick/sage/finance/opentick.pyx:
31:18: Syntax error in C variable declaration
sage: There was an error installing modified sage library code.
observations:
1) The OTClient class has a strange declaration: "class DLL_EXP
OTClient { public: ... }".
2) I poked around for this DLL_EXP guy. There's a declaration in
OTDef.h that is simply "#define DLL_EXP"
3) OTClient.h includes "<vector>". The Cython wiki says that this
should be handled in a particular way.
After some poking around on the web, I found that this DLL_EXP define
is used to perform callbacks to DLL files. Here's a reference:
http://bcbjournal.org/articles/vol3/9902/Using_callbacks_in_DLLs.htm?PHPSESSID=667cfb11c3fca48da9150a0b667fa67b
Trying to figure out what this means for wrapping in Cython. Any
thoughts? I'm a bit stumped at the moment.
--
Chris
Glenn H Tarbox, PhD
This all smells like windows stuff and can likely be yanked for now.
We'll work it when we need to be cross platform.
William Stein
>
> Hello,
>
> Just a little update on my progress:
>
> I established basic communication between the OTFeed library and my
> opentick.pyx source file, meaning that compiling works with the
> command
Is your code posted somewhere? I could try hacking on it a little
when I have access to it.
-- William