IBM Coursera Deep Learning Week 3 Data Size Problem

3 views
Skip to first unread message

ritch...@abrs.com.hk

unread,
Jul 18, 2018, 10:47:40 AM7/18/18
to Machine Intelligence and Data Science WonderCorp

I tried to run through the demo notebook cwt_elt.ipynb with some interesting problems:

(my notebook link attached) https://dataplatform.cloud.ibm.com/analytics/notebooks/v2/d3d09c05-e9af-47e5-9d10-ce175d25ef03/view?access_token=89c33a299bde9ae62274f3865ef2f6d1269685878193dc4a50d9610e789dd7cd

after executing > result_healthy = read_folder('./cwr_healthy/'

  1. Size of result_health, which is a numpy_array, a result of grapping 3 files with fileID #98, 97, 100,  I checked the following sizing: (> just anotate  jupyter cell input plus ouput, not any command prompt)

> result_healthy.shape  (1213484, 3)

> re sult_healthy.shape[0]   3

> result_healthy.shape[1]    3

> result_healthy  #containing 98, 97 and 100 file id  (99 was skipped due to size mismatch)

array([['98', '0.046104', '0.023216363636363632'],
       ['98', '-0.03713353846153846', '0.08115454545454545'],
       ['98', '-0.08949599999999999', '0.09533090909090909'],
       ...,
       ['100', '0.10930181818181818', '-0.02357353846153846'],
       ['100', '0.05218545454545454', '0.005215384615384615'],
       ['100', '-0.00452', '0.047772923076923074']], dtype='|S32')

2. It is strange that .shape show a size of 1213484, i guessed is  close to 3 x  485643, equal to 3 times  mlf['X100_DE_time'].size if all three files are fairly equal in size, which would be approx the sum of the number of 'rows" in 98, 99, and 100 files processed.

However the shape[0] and shape[1] are 3, 3 respectively. 

a. First it is strange why shape[0], shape[1]. and .shape gave inconsistent output?

b. Secondly, result-healthy.shape, at a glimpse, should have a dimension of its total length say 1213484 x 3, not 3 x 3, unless each 98, 99, 100 record groups are enclosed with another [  ...]  (or something equivalent as I am not too familiar of python ndarray structure), 

Is my understanding of ndarray causing a problem,or there are other hidden reasons?


Reply all
Reply to author
Forward
0 new messages