dtw_subsequence causing python to crash (32 bit, python 2.7)

[Bo Lu]

Hi All,

I am trying to write a function to search a time series data for similar trends occurring in the past. The subsequence DTW warping function in MLPY saves me a lot of effort to write my own code. However, I am encountering a problem with python crashing when the second longer vector (y) gets large. I suspect this has to be with the memory allocation used in the dtw but do not know how to fix it myself (I checked that the array lengths are passed as integers, which should have a max value of 2147483647, so i don't understand why it can't handle a 265k element array). Could anyone help me?

It works fine with a 300 element x and a 6000 element y vector

In [7]:

x = np.array(signature)
y = np.array(df.ix['2015-04-30':])
x.shape
y.shape

Out[7]:

(6147, 1)

In [8]:

dist, cost, path = dtw_subsequence(np.squeeze(x), np.squeeze(y))
print dist
print path[1][0]
print path[1][-1]

45.11449452
1540
1743

But the following code would cause my python to crash:

In [23]:

x = np.array(df.ix['2015-04-29 6 pm':'2015-04-29 10 pm'])
y = np.array(df2.ix['2014'])
#results =ranked_subDTW(x[:,0],y[:,0])   

In [26]:

print y.shape
print x.shape

(265020, 1)
(300, 1)

In [ ]:

dist, cost, path = dtw_subsequence(x[:,0],y[:,0])

Crash information:

Problem signature:

  Problem Event Name: APPCRASH

  Application Name: python.exe

  Application Version: 0.0.0.0

  Application Timestamp: 56311b83

  Fault Module Name: dtw.pyd

  Fault Module Version: 0.0.0.0

  Fault Module Timestamp: 4f5e05de

  Exception Code: c0000005

  Exception Offset: 000012c6

  OS Version: 6.1.7601.2.1.0.256.4

  Locale ID: 1033

  Additional Information 1: 0a9e

  Additional Information 2: 0a9e372d3b4ad19135b953a78882e789

  Additional Information 3: 0a9e

  Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

Thanks,

Bo