Parallel Processing for Running Optimizations

[Mick O'Donnell]

Hi all,

Just wondering if anyone has looked at use parallel processing for optimizing strategies yet. I've been using the NVIDIA's CUDA library for other heavy lifting jobs with good results. There's a project called JCUDA which supplies a Runtime API with Java bindings for CUDA. If no one has written anything for this yet, I may take it on as a side project.

http://www.jcuda.org/

regards,

Michael

[Klaus]

Hi, 

I was just wondering: was there any further efforts to do CUDA-based backtesting for JBT?

Cheers

 Klaus

Am Donnerstag, 18. April 2013 03:44:58 UTC+2 schrieb Sheb Latsama:

I am looking into this, but it is still far from ready for prime time.  I have a good speedup, but for some reason the results are diverging from the expected values in seemingly random ways.  I will update the group when I have something ready to share... it is still 'pre-alpha' at this point.

I asked Eugene to remove some info I shared with him on this topic as it is premature at this point, and he was kind enough to do so.

[Judson Wilson]

I think it would be more worthwhile to do a cloud based multi-computer approach.

--
You received this message because you are subscribed to the Google Groups "JBookTrader" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jbooktrader...@googlegroups.com.
To post to this group, send email to jbook...@googlegroups.com.

Visit this group at http://groups.google.com/group/jbooktrader.
For more options, visit https://groups.google.com/d/optout.

[Klaus]

Your mileage may vary. 

For me, such a solution might mean a speedup of about 30 (given the number of graphics cores, the differences in performance, non-optimal parallelization, etc. - but it is a rather wild, but conservative guess). 

If you look at pricing on the cloud, using this might cost like 5-10 USD / hour (were also partial hours count). 

Given the number of runs one has to do to reach a reasonable result, this can easily go to dozens/hundreds of hours. This might be ok from your perspective, it seems like a bad deal from my point of view. (this is also why modern high-performance computers often take a GPU heavy approach)

So, I think, going GPU first is probably a good idea, then one might want to cascade it later with a multicomputer approach (AWS offers specific GPU instances for the cloud). 

But all this is mute, the question is whether any one with the necessary expertise is pursuing this approach. 

Or perhaps someone has better data for estimating the kinds of results to be expected. 

Klaus

[Judson Wilson]

I didn't notice this was in the original thread. A factor of 1000x improvement? REALLY? I have a hard time believing it.

[Klaus]

Not sure where you take the 1000x improvement from? I was talking about a 30x - and this was an ad hoc guess.

If you look around, there are comparisons which make this seem realistic (e.g., http://www.hs.uni-hamburg.de/DE/Ins/Per/Abhranil/summer_report.pdf), but this also shows it dependents very much on the task specifics. This would be the interesting question here, whether such a speed-up would be realistic.

(there the border is at a factor of 400, but this obviously depends on the specifics of the CPU and GPU compared)

[Judson Wilson]

Quoting Eugene:

> Riunning the JBT optimization on he high end GPU (AMD Radeon 7900 series) indicates a speedup in the order of 1000 times faster. The big drawback is that it required a complete re-write of the multiple JBT classes, essentially flattening the object-oriented model into array processing...