Dmitry's take on tradelink published: tradelink.afterlife

[Dmitry Shevkoplyas]

hi all,

It seems I managed to rebuild C++ TWSServer (it is part of tradeling - IB connector) to use latest IB API.

It is not tested yet, since IB has maintenance day today and TWS can not even start, but it is compilable and runable :)

https://github.com/shevkoplyas/tradelink.afterlife

tradelink's responses in published version can take parameters from external JSON files, also if you use "MessageBusableResponseTemplate" class

like in this file "Responses/_TS_step_by_step.cs":

    public class _TS_step_by_step : MessageBusableResponseTemplate

then your response will also emit all the events to RabbitMQ message bus :)

(just experimenting to be able to use external viewers with tradelInk.afterlife, since it does not have viewers, I'd say at all:)

I'll put more info in some readme's later when things are more stable/tested. (don't have much free time). For now the whole thing is just my "as is" take on TL.

--
с ч.гл.ув.,
Дима Ш.

[Letian]

Hi,

I also compiled the IB connector with tradelink with link below. Hopefully it's a bit helpful to the project. Thanks.

# Non-members may check out a read-only working copy anonymously over HTTP.
svn checkout http://my-ib-quant-trading.googlecode.com/svn/trunk/ my-ib-quant-trading-read-only

Letian

[Dmitry Shevkoplyas]

Hi Letian,

so far you are the only developer I heard of who can put together few lines of code for tradelink!  : )

I sort of lost the hope to see TradeLink supported by some other than PracPlay group of developers and decided to go hard way "all my myself" : )
https://github.com/shevkoplyas/tradelink.afterlife

seems you are on the similar path?

Let us share the current "state of the art" and try to find some common points, and hopefully it will be win-win for both of us.

Few words about myself:

- I'm 38, full time software developer (mostly under linux: ruby, c/c++, now due to TL some C# :), windows/linux friendly

- my broker is IB and I almost @25K on my account, which is the threshold for daydrading (if you own less then 25K then you limited to ~3-4 buy/sell order a day

- I've paper account (with 1M on it:)

- I've lots of ideas to try, but until now didn't have good framework

- I'd love to get lots of historical TIK files for lots of backtesting of my ideas :), for example I'm ready to write my own application to get 1 yr historical data via IB API, but they provide only down to 30sec bars (which is ok for some ideas and won't work for others - still better than nothing)!  btw I used "bar to TIK" converter back in a day (not sure if it is part of tradelink or not) it converted each bar into 4 ticks (one for "open" one for "high", for "low" and "close" - 4 ticks total)

- I'd love to pump historical data with 5-6 other people (more - better:) and then share results with them so we have all the data we need by doing small effort (each of participants)

- or we can (as a group) run realtime responses and collect TIK files for different tickers and then share for backtesting. Or may be we can buy some sort of DVD 10yrs+ of ticks for s&p 500 or something.

- I dont have much free time for TL (only up to 2-3 hours a day) due to my big family (3kids: 3.5yr twins + 1.5yr boy) - lots of fun!-)

may be if we dig TradeLink together we'd have less steep learning/developing curve and can improve/bugfix the whole thing much more efficiently!

I finished with TWSServer (connector for IB) for now and now I'm trying to figure out how to subscribe on tickers properly (example: if I"m trying to subscribe for ABX, IBM, GOOG, MFC, AAPL, then only 2 of 5 will actually deliver ticks to my response and I'm trying to figure out why).

Also, since most of the time I'm working with TL after whole family go to bed, the markets are closed. I found a good workaround for myself: I'll test my bots on Forex tickers (afterall does it really matter what to trade?-)), it would be good for real-time debugging afterhours! But I can't subscribe to CAD.JPY pair from tradelink. In 1-2 days (read nights), I will figure out why and how.

gotta run. . .
talk to you later!-)

 

--
с ч.гл.ув.,
Дима Ш.

On Fri, Nov 15, 2013 at 8:31 AM, Letian <leti...@gmail.com> wrote:

Hi,

I also compiled the IB connector with tradelink with link below. Hopefully it's a bit helpful to the project. Thanks.

https://hotfile.com/dl/255139436/2ca43dd/IBTradelink.zip.html

Letian

On Saturday, November 2, 2013 3:04:51 PM UTC-4, shevkoplyas wrote:

--
You received this message because you are subscribed to the Google Groups "TradeLink-Free" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tradelink-fre...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Wilhelm Stroods]

Dmitry,

Do you have a build of Tradelink with your mods to remove the PracPlay tracking logic?

the latest build I have is 3754.07-Aug-2013.

Let's rename your build with that fix to be 4000.00  then we can post it on the new site.

Please send location of your latest build.

Best Regards,

James

[Dmitry Shevkoplyas]

Hi Wilhelm,

[Dmitry's build]

My take on TL is published here: 

https://github.com/shevkoplyas/tradelink.afterlife

Please feel free to do whatever you thing necessary to it.

But "afterlife" has several changes which you'd be aware of before you decide what to publish:

    - I deleted all connectors except IB (which is as you can guess my broker:)

    - I modified TWSServer alot. Now it reflects the latest IB API

      Also I modified TWSServer dialog window, which had only 1 non-resizable CEdit field (btw with only 30Kbyte text limit). Not it has 5 auto-resizable rich edit (no text limits) boxes to show all the messages flying between:

    IB API -> Server

    IB API <- Server

                   Server -> Clients (ASP, Quotopia etc)

                   Server <- Clients (ASP, Quotopia etc)

 

So TWSServer is kinda latest, but has more of "for developers" view than "for end customer" view.

    - I added new dependencies which 99% of customers probably will never need:

      RabbitMQ (as a message bus)

      MongoDB (as a storage for JSON objects)

      JSON parser (to understand JSON)

      Something (don't remember the name now) to parse command line options.

      You can find all these binaries in _dependencies folder and some projects (Kadina, ASP, Responses) have references to them.

I'd suggest for your TL to take only TWSServer from me as is, but not the rest (btw TWSServer is a separate solution, it has its own .sln file and it is C++, not C#). And then just tweak few lines to disable tracking as I described earlier. Though some of the features I added are useful and cool. Like command line option parser and being able to provide settings for strategies as JSON files (good for automatization, good readability etc.) Emitting messages to the message bus an having other distributed subscribers to it is even cooler, but will make TL use harder for common (non programmers) user. So think twice what you need and what not. I'm keep developing "afterlife" heavily.

[next build number]

since PracPlay still developing their own TL, I'd suggest to give you TL different name, for example "FreeTradeLink" or "TradeLink.Free" or alike and start from rev.#1

Regards,

Dmitry

[Letian]

Hi Dmitry,

 

Thank you very much for the introduction. I can see similar background we share (except for the kids, finger crossing). I focus on only a couple of index etfs and futures and I'm able to download 1 sec bars from IB. I remember TL is missing a letter 's' maybe that's why you couldn't download 1 sec bars. You can confirm it with my c# interface/tester. In terms of real time subscription, I don't find a problem subscribing 5 names at a time.

 

I am also on the paper trading/preparing stage. I think team work is a good idea, hopefully we can have a dedicated group to share ideas and experiences.

 

Best,

Letian

[Wilhelm Stroods]

Thanks Dmitry for that detailed explanation.

I will update the most recent build to remove PracPlay's tracking code across all modules. then will post that version as version #1 of TradeLink Software.

Would like to include your IB logic as well for subsequent release.

Regarding strategies for trading, we can add some threads on the bulletin board on the new site for that.  Will do it. 

talk soon,

James

[Dmitry Shevkoplyas]

hi all,

update on "tradelink.afterlife" branch:

(quote from known-issues.txt)

This file describes known issues for tradelink.afterlife

--------------------------------------------------------

- when close ASP window there's still ASP process running in a background

  tmp workaround: run/stop ASP using visual studio (not by running compiled ASP.exe)

- when close ASP which run response and response was subscribed to some real-time

  ticks, then TWSServer still holds the subscription for that ticker and sends

  GotTick() (to nowhere)

  Now, if you start ASP + response and try to subscribe to same ticker, then

  asp.cs::tl_gotTick() never fires (never receives ticks) and your response

  wont receive ticks (since ASP wont ever call response::GotTick())

  tmp workaround: restart both IB connector and ASP + response - solves it for now

- for "after hours" real time debug (after 16:00 EST) we need to subscribe to

  something actively trading 24/7, like forex CAD.JPY.

  ASP + response will not subscribe to forex ticker CAD.JPY because exchange

  is set by default is set to SMART instead of "IDEALPRO" for that ticker

  as well as security type defaults to STK instead of CASH for this case.

  tmp workaround (works only for CAD.JPY for now):

C:\tradelink.afterlife\BrokerServers\TWSServer\TWS_TLServer.cpp

if (sec.sym == "CAD.JPY"){

contract.exchange = "IDEALPRO";//"SMART";

//contract.secType = CASH;

sec.type = TypeFromExchange(contract.exchange);

}

  works!  :)

  note: when subscribing to CAD.JPY it is case sensitive, so "cad.jpy" won't fly

(end of quote)

[Carlos Huergo]

Hi, Dmitry, I was a ´pasive' follower of tradelink. Did you finally formed another group improving or mantaining the free afterlife tradelink?

Regards, Carlos

2013/11/21 Dmitry Shevkoplyas <shevk...@gmail.com>

--
Carlos Huergo

[Dmitry Shevkoplyas]

Hi Carlos,

[The short answer]

No and I'm not trying to. 

James (wilhelm...@gmail.com) is trying to put it all back together: creating a web site + group of developers to support TradeLink, but so far I didn't saw any major "new versions" from that direction.

[A bit longer answer]

I would start from the question to everybody:

[Q] what was very good and is holding you to use TradeLink? what kind of functionality does TradeLink provide for you that you can't find anywhere else?

for me the answer was: 

    - gives me IB connector which was woking out of the box

    - calls my callback functions (like "got tick", "got order" etc.)

    - stores TIK files

    - can replay TIKs for my backtest experiments

That is it! 

That is all I used (TA-lib functionality is a separate project, so I didn't mentioned it here)

[Q] what was not good about TradeLink experience?

again my imho here was:

    - no charting at all!

    - works only on windows (I want to move trading bot to Amazon cloud Ubuntu boxes)

    - no more support from PracPlay (this is really big one!)

    - is quite complex solution with very limited  capabilities

    - now way to easily see what is going on during live trading

      (ideally this functionality should also be available remotely)

      (also it should be possible to see real-time stats per each individual response running even if two (or more) responses are trading same ticker)

by the last item I mean: if you look deeper into IB connector, then you'll notice bunch of functionality is not implemented. I'll just list some questions which you CAN'T make to work (without code modification) using existing "latest TradeLink" snapshot:

    - how do resolve ambiguity for some tickers (like MSFT)

    - how do you subscribe to tickers not SMART-routed? like for example forex pairs CAD.JPY should use "IDEALPRO" instead.

    - how do you debug "failed to subscribe" cases for some securities - there's no

      visibility of why it failed and where particular it failed (and you need to be able to fix it yourself)

    - how do you subscribe to historical data? for example you algo-trading bot could get last 3 days 5sec bars to 

    - how do you trade options?

    - how do you get full option chains for some stocks

    - what do you do when IB publish new TWS API (which they do from time to time)?

The root of all events is broker's API (since all the rest is just layers of additional complexities on top of that), so few months ago I joined TWA API yahoo group (I posted detailed info about that group + forwarded few posts here from that group few weeks ago).

Now I have 2 versions of own C++ connectors which is 100 times simpler and it already gives me:

    - IB connector with all functionality at my fingertips

    - calls my callback functions (like "got tick", "got order"...)

As you can see - this is already more than 50% of what I need :)

For now it does not storeTIK files for me, but as you can understand it is very easy to add.

To make "replay" might be more tricky, but also doable.

Also on the good side I have now:

    - decreased complexity (about 1.6K-1.7K lines of cpp code! compare to ~200K lines in cs files of TradeLInk!!) 

    - much less moving parts (no longer invisible windows exchanging serialized structures)

    - posix compatible i.e. I can compile/run the solution in windows, Linux (Ubuntu) and OS-X !

So for now I'm abandoning TradeLink as a whole piece. It was nice toy and it was good milestone on the way to algo-trading,

but it is too limited and heavy without 24/7 support from some team.

Next steps for me would be to:

    - add ZeroMQ messaging into the mix after I'm completely happy with what I've got so far. In this case 

    - add some visualizer

    - add ability to see what's going on remotely

Regards,

Dmitry Sh.

[Wilhelm Stroods]

Hello guys,

I have most of it done, but am at the same time, working on getting a web version up and running, so trading system developers can post their systems for lease, and non-technical users can use trading systems without having to code themselves.  

Soon will send details on the new tradelink site.    stay tuned.

James

[Carlos Huergo]

Amazing Dmitry!

Congratulations!

Really having an IB connector with the main events is more than 50%. I also follow the TWS API yahoo group under other email, but I was "experimenting" with tradelink as C# more comfortable for me then C++, and it had a lot of functionalities to copy and paste. 

I did not like their "spying" capabilities, but...

Regards

Carlos 

2013/12/14 Dmitry Shevkoplyas <shevk...@gmail.com>

--
Carlos Huergo