makeSOCKcluster anomaly

40 views
Skip to first unread message

Whitney

unread,
Feb 6, 2011, 8:44:50 PM2/6/11
to Leverage Space Trading
Any idea why this is blowing up or what to do about it? The full
history follows my signature below.

A few observations:

1) Apparently SOCK uses "enquote" internally. There's a "masked
from 'package:base' message in the upper half of the history. Don't
know what that means. I've run package updates a couple of times and
no updating is apparent.

2) The script seems to hang on or just after this line
clust <- makeSOCKcluster(2)

3) Identical behavior on x64 and 32-bit R 2.12.1.

4) System has 4 cores. Same symptom with makeSOCKcluster(2) and
makeSOCKcluster(4).

Grateful for a trailhead....

Whitney
_____________________
> Sys.time()
[1] "2011-02-06 18:41:24 MST"
> outcomes <- cbind(
+ c(-8553,405.417,10223.474,28457.75,37631,56502))
>
> probs <- c(
+
0.006666666667,0.686666666667,0.253333333333,0.026666666667,0.013333333333,0.013333333334)
>
> library(LSPM)
> library(snow)
>
> #How many cores SOCKcluster will drive into snow
> corecount <- 4
>
> port <- lsp(outcomes,probs)
>
> #data(port)
>
> # DEoptim parameters (see ?deoptim)
> DEctrl <- list(NP=30, itermax=100)
>
> # Unconstrainted Optimal f
> res <- optimalf(port, control=DEctrl)
Iteration: 1 bestvalit: -1.335943 bestmemit: 0.975721
Iteration: 2 bestvalit: -1.336114 bestmemit: 0.972867
Iteration: 3 bestvalit: -1.336114 bestmemit: 0.972867
Iteration: 4 bestvalit: -1.336182 bestmemit: 0.970177
Iteration: 5 bestvalit: -1.336182 bestmemit: 0.970177
Iteration: 6 bestvalit: -1.336188 bestmemit: 0.968866
Iteration: 7 bestvalit: -1.336188 bestmemit: 0.968905
Iteration: 8 bestvalit: -1.336188 bestmemit: 0.968905
Iteration: 9 bestvalit: -1.336188 bestmemit: 0.968905
Iteration: 10 bestvalit: -1.336188 bestmemit: 0.969201
Iteration: 11 bestvalit: -1.336188 bestmemit: 0.969010
Iteration: 12 bestvalit: -1.336188 bestmemit: 0.969017
Iteration: 13 bestvalit: -1.336188 bestmemit: 0.969017
Iteration: 14 bestvalit: -1.336188 bestmemit: 0.969070
Iteration: 15 bestvalit: -1.336188 bestmemit: 0.969070
Iteration: 16 bestvalit: -1.336188 bestmemit: 0.969064
Iteration: 17 bestvalit: -1.336188 bestmemit: 0.969064
Iteration: 18 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 19 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 20 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 21 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 22 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 23 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 24 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 25 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 26 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 27 bestvalit: -1.336188 bestmemit: 0.969060
Iteration: 28 bestvalit: -1.336188 bestmemit: 0.969060
Iteration: 29 bestvalit: -1.336188 bestmemit: 0.969060
Iteration: 30 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 31 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 32 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 33 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 34 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 35 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 36 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 37 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 38 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 39 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 40 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 41 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 42 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 43 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 44 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 45 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 46 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 47 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 48 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 49 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 50 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 51 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 52 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 53 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 54 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 55 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 56 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 57 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 58 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 59 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 60 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 61 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 62 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 63 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 64 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 65 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 66 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 67 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 68 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 69 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 70 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 71 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 72 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 73 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 74 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 75 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 76 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 77 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 78 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 79 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 80 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 81 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 82 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 83 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 84 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 85 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 86 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 87 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 88 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 89 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 90 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 91 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 92 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 93 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 94 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 95 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 96 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 97 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 98 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 99 bestvalit: -1.336188 bestmemit: 0.969059
Iteration: 100 bestvalit: -1.336188 bestmemit: 0.969059
>
> # Margin-constrainted Optimal f
> resMargin <- optimalf(port, control=DEctrl, equity=1e5, margin=-port$maxLoss*2)
Iteration: 1 bestvalit: -1.209148 bestmemit: 0.495248
Iteration: 2 bestvalit: -1.209868 bestmemit: 0.497365
Iteration: 3 bestvalit: -1.209868 bestmemit: 0.497365
Iteration: 4 bestvalit: -1.209868 bestmemit: 0.497365
Iteration: 5 bestvalit: -1.209868 bestmemit: 0.497365
Iteration: 6 bestvalit: -1.209868 bestmemit: 0.497365
Iteration: 7 bestvalit: -1.210517 bestmemit: 0.499278
Iteration: 8 bestvalit: -1.210517 bestmemit: 0.499278
Iteration: 9 bestvalit: -1.210693 bestmemit: 0.499795
Iteration: 10 bestvalit: -1.210693 bestmemit: 0.499795
Iteration: 11 bestvalit: -1.210756 bestmemit: 0.499982
Iteration: 12 bestvalit: -1.210756 bestmemit: 0.499982
Iteration: 13 bestvalit: -1.210756 bestmemit: 0.499982
Iteration: 14 bestvalit: -1.210756 bestmemit: 0.499982
Iteration: 15 bestvalit: -1.210756 bestmemit: 0.499982
Iteration: 16 bestvalit: -1.210756 bestmemit: 0.499982
Iteration: 17 bestvalit: -1.210756 bestmemit: 0.499982
Iteration: 18 bestvalit: -1.210756 bestmemit: 0.499982
Iteration: 19 bestvalit: -1.210760 bestmemit: 0.499995
Iteration: 20 bestvalit: -1.210760 bestmemit: 0.499995
Iteration: 21 bestvalit: -1.210760 bestmemit: 0.499995
Iteration: 22 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 23 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 24 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 25 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 26 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 27 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 28 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 29 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 30 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 31 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 32 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 33 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 34 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 35 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 36 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 37 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 38 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 39 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 40 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 41 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 42 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 43 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 44 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 45 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 46 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 47 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 48 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 49 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 50 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 51 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 52 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 53 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 54 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 55 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 56 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 57 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 58 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 59 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 60 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 61 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 62 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 63 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 64 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 65 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 66 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 67 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 68 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 69 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 70 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 71 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 72 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 73 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 74 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 75 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 76 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 77 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 78 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 79 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 80 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 81 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 82 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 83 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 84 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 85 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 86 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 87 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 88 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 89 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 90 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 91 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 92 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 93 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 94 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 95 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 96 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 97 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 98 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 99 bestvalit: -1.210762 bestmemit: 0.500000
Iteration: 100 bestvalit: -1.210762 bestmemit: 0.500000
>
> # Create snow socket cluster for two cores
> library(snow)
> clust <- makeSOCKcluster(2)

>
> # Drawdown-constrained Optimal f using two cores
> resSnow <- optimalf(port, probDrawdown, 0.1, DD=0.2, horizon=4, snow=clust, control=DEctrl)
Error in function (lsp, DD, horizon, calc.max = 10, error = 0.001,
sigma = 3, :
object 'clust' not found
>
> Sys.time()



Whitney

unread,
Feb 8, 2011, 12:50:23 AM2/8/11
to Leverage Space Trading
Euh, it's even worse. I surfed around for a troubleshooting
suggestion, which is to have makeSOCKcluster record a log file. Even
that's not being performed.

Here's that that option looks like:

clust <- makeSOCKcluster(corecount, outfile="C:/windows/temp/
snow.log")

Whitney

unread,
Feb 8, 2011, 1:31:28 AM2/8/11
to Leverage Space Trading
Surfed around and found this alternative, which is working for now:

clust <- makeCluster( 2, type = "SOCK")

Apparently this behavior appears mainly on Windows after R 2.10.
Here's a relevant thread: <https://stat.ethz.ch/pipermail/r-sig-hpc/
2010-August/000721.html>
> ...
>
> read more »

Whitney

unread,
Feb 8, 2011, 10:05:58 PM2/8/11
to Leverage Space Trading
With this working, I noticed that cpu utilization wasn't very high for
the cluster processes. I increased the number of processes to see the
effect on completion time of calculations of constrained f on a single
marketsystem. This is a 4-core Win7x64 box.

ProcCount,MinutesToComplete
2,33
4,34
8,36
16,41
32,51

Cranking up the number of processes in the SOCK cluster doesn't help.
> ...
>
> read more »

Michael White

unread,
Feb 9, 2011, 3:54:00 AM2/9/11
to leverage-sp...@googlegroups.com
The limitation may be the available cache on your CPU(s) or the RAM. If the cache is overflowing out to RAM, and possibly even the RAM out to disk, then increasing the number of processes will only aggravate the situation, as you end up with more contention for the same limited resource.

Mike

--- On Tue, 2/8/11, Whitney <whitne...@gmail.com> wrote:



____________________________________________________________________________________
The fish are biting.
Get more visitors on your site using Yahoo! Search Marketing.
http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php

Whitney

unread,
Feb 10, 2011, 12:35:25 AM2/10/11
to Leverage Space Trading
Sounds plausible.

I'm going to run some tests with 2 marketsystems and see how the
behavior differs. I wonder what sort of difference using a different
type of cluster would make, if any?

OBTW, Michael: I just recognized your handle from the AB group on
Yahoo. Are you available to discuss (offline) an LSP/Amibroker
implementation? Joshua and I discussed the possibility briefly several
months ago. I've been thinking about posting a general message to the
Yahoo list but would rather kick the idea around 1:1 first.

On Feb 9, 1:54 am, Michael White <sfclimb...@yahoo.com> wrote:
> The limitation may be the available cache on your CPU(s) or the RAM. If the cache is overflowing out to RAM, and possibly even the RAM out to disk, then increasing the number of processes will only aggravate the situation, as you end up with more contention for the same limited resource.
>
> Mike
>
> --- On Tue, 2/8/11, Whitney <whitneybro...@gmail.com> wrote:
> ...
>
> read more »

Michael White

unread,
Feb 10, 2011, 4:15:14 AM2/10/11
to leverage-sp...@googlegroups.com
Yes, I'm available. You can contact me at this email address.

I've been planning to do something a little different than a pure AmiBroker implementation. I'm approaching an alpha release of a distributed backtester/optimizer/walk forward analysis framework.

At the moment the product is a desktop application that spawns multiple instances of a trading engine across multiple cores of one or more machines on a LAN, capturing all results in a database. Once the desktop version is shipping, I'm seriously considering releasing a cloud edition hosted on the web for on-demand use.

The application design is entirely modular and should allow supporting any trading engine that can be programatically controlled. Similarly, it allows for integrating 3rd party optimizers, including something like leverage space.

Currently I've only implemented the integration for AmiBroker and AmiBroker's optimizers. However, I definitely plan to provide a leverage space optimizer. Any optimizer written using the AmiBroker SDK will work in my environment "out of the box", though it will not take full advantage of the parallelization that I offer unless making a slight change to return multiple combinations in a single request, as opposed to a single combination per request. In the case of the CMA-ES optimizer this was easily done, required no API change, and remains 100% backwards compatible with AmiBroker (i.e. revised DLL can still be used in both AmiBroker and my product).

Where it may not be of help to you is that I plan to write the leverage space optimizer in Java instead of C++, so it won't be available as a DLL to AmiBroker. My motivation for that is not to deny AmiBroker users, nor to force anyone into my own product, but rather simply because I haven't worked in C++ for 15 years and don't have any interest in going back to it. I'm waaaaay more productive in Java and don't want to waste time getting up to speed again in another language, development tool, etc.

Mike

--- On Wed, 2/9/11, Whitney <whitne...@gmail.com> wrote:

Joshua Ulrich

unread,
Feb 10, 2011, 11:02:13 AM2/10/11
to leverage-sp...@googlegroups.com
I agree with Mike's hypothesis, but I would like to offer another
potential cause: communication between the nodes. For example:

# Code
require(LSPM)
data(port)

require(snow)
cl <- makeSOCKcluster(2)

### Short horizon
system.time(for(i in 1:1e3) probDrawdown(port, 0.1, 3, error=1e-9))
# user system elapsed
# 0.62 0.00 0.63
system.time(for(i in 1:1e3) probDrawdown(port, 0.1, 3, error=1e-9, snow=cl))
# user system elapsed
# 0.89 0.05 1.47

### Long horizon
system.time(probDrawdown(port, 0.1, 7, error=1e-9))
# user system elapsed
# 30.94 0.00 31.00
system.time(probDrawdown(port, 0.1, 7, error=1e-9, snow=cl))
# user system elapsed
# 0.00 0.00 15.59

stopCluster(cl)
# End Code

The short horizon example takes longer with snow because it takes time
to send data to/from the nodes to the master process. The nodes spend
most of their time waiting (as evidenced by the massive traffic on the
loopback interface on my *nix machine). The long horizon example is
faster because each node spends the majority of the time pegging the
dual-core CPU at 50%.

Best,
--
Joshua Ulrich  |  FOSS Trading: www.fosstrading.com

Whitney

unread,
Feb 10, 2011, 10:42:34 PM2/10/11
to Leverage Space Trading
OK, Mike, I'll follow up off-line

On Feb 10, 2:15 am, Michael White <sfclimb...@yahoo.com> wrote:
> Yes, I'm available. You can contact me at this email address.
>
> I've been planning to do something a little different than a pure AmiBroker implementation. I'm approaching an alpha release of a distributed backtester/optimizer/walk forward analysis framework.
>
> At the moment the product is a desktop application that spawns multiple instances of a trading engine across multiple cores of one or more machines on a  LAN, capturing all results in a database. Once the desktop version is shipping, I'm seriously considering releasing a cloud edition hosted on the web for on-demand use.
>
> The application design is entirely modular and should allow supporting any trading engine that can be programatically controlled. Similarly, it allows for integrating 3rd party optimizers, including something like leverage space.
>
> Currently I've only implemented the integration for AmiBroker and AmiBroker's optimizers. However, I definitely plan to provide a leverage space optimizer. Any optimizer written using the AmiBroker SDK will work in my environment "out of the box", though it will not take full advantage of the parallelization that I offer unless making a slight change to return multiple combinations in a single request, as opposed to a single combination per request. In the case of the CMA-ES optimizer this was easily done, required no API change, and remains 100% backwards compatible with AmiBroker (i.e. revised DLL can still be used in both AmiBroker and my product).
>
> Where it may not be of help to you is that I plan to write the leverage space optimizer in Java instead of C++, so it won't be available as a DLL to AmiBroker. My motivation for that is not to deny AmiBroker users, nor to force anyone into my own product, but rather simply because I haven't worked in C++ for 15 years and don't have any interest in going back to it. I'm waaaaay more productive in Java and don't want to waste time getting up to speed again in another language, development tool, etc.
>
> Mike
>
> ...
>
> read more »

Whitney

unread,
Feb 10, 2011, 10:43:23 PM2/10/11
to Leverage Space Trading
Joshua,

Thanks. I'll vary the horizon and see what happens.
> > --- On Tue, 2/8/11, Whitney <whitneybro...@gmail.com> wrote:
>
> >> From: Whitney <whitneybro...@gmail.com>
> ...
>
> read more »

Whitney

unread,
Mar 7, 2011, 11:33:51 PM3/7/11
to Leverage Space Trading
I've results similar to Joshua's, following my signature below.
That's with his code model.

Separately, results not shown here, I ran a 2 marketsystem hrz 5
test. Results were similar. The test took about 6 hours with 2 cores
specified for SOCK, about 9 hours with 4 cores specified.

So the same behavior is happening with 1 marketsystem and 2
marketsystems in test. With a longer horizon the time efficiency of
more cores appears to kick in.

Mike, unless you think these results zero out your hypothesis, I'd
like to test it, too, but I'm pretty sure I don't have the proper
config of system monitoring tools.

Best,
Whit

> # LSPM performance tests
>
> #Joshua's code
> require(LSPM)
> data(port)
> require(snow)
> cl <- makeSOCKcluster(2)
> ### Short horizon
> system.time(for(i in 1:1e3) probDrawdown(port, 0.1, 3, error=1e-9))
# user system elapsed
# 0.37 0.00 0.39
> # user system elapsed
> system.time(for(i in 1:1e3) probDrawdown(port, 0.1, 3, error=1e-9, snow=cl))
# user system elapsed
# 1.24 0.09 2.04 # overhead clobbers performance
> # user system elapsed
>
> ### Long horizon
> system.time(probDrawdown(port, 0.1, 7, error=1e-9))
# user system elapsed
# 26.74 0.01 26.88
>
> system.time(probDrawdown(port, 0.1, 7, error=1e-9, snow=cl))
# user system elapsed
# 0.00 0.00 16.86 # more cores clobber overhead
>
> stopCluster(cl)
> # End Code
>
> #Joshua's code again, with 4 instead of 2 for sock
> require(LSPM)
> data(port)
> require(snow)
> cl <- makeSOCKcluster(4)
> ### Short horizon
> system.time(for(i in 1:1e3) probDrawdown(port, 0.1, 3, error=1e-9))
# user system elapsed
# 0.4 0.0 0.4 #marginally longer than before
>
> system.time(for(i in 1:1e3) probDrawdown(port, 0.1, 3, error=1e-9, snow=cl))
# user system elapsed
# 1.62 0.12 2.29 # about 10% worse than before
>
>
> ### Long horizon
> system.time(probDrawdown(port, 0.1, 7, error=1e-9))
# user system elapsed
# 26.77 0.00 26.83 #very close to 2-core result
>
> system.time(probDrawdown(port, 0.1, 7, error=1e-9, snow=cl))
# user system elapsed
# 0.00 0.00 9.08 #nice long-horizon improvement vs. 2 cores
>
> stopCluster(cl)
>
> # end LSPM performance tests
> ...
>
> read more »
Reply all
Reply to author
Forward
0 new messages