Loading pickled files

[James Morrill]

Hello,

Can anyone help me with my error:

Warning:
  One or more of the files submitted as part of this entry do not
  appear to have free/open-source licenses.  PhysioNet Challenge
  entries must consist entirely of free and open source software; if
  your entry is found to contain proprietary files, it will be
  disqualified from winning challenge awards.

  (This is an automated message based on the presence of certain key
  words in your entry's source code.  If these files are in fact
  freely licensed, feel free to ignore this message.  There is no need
  to re-submit your entry.)

  Unknown binary files (5):
	models/experiments/challenge/final/1/train_fm_1.pickle
	models/experiments/challenge/final/1/train_fm_2.pickle
	models/experiments/challenge/final/1/xgb.pickle
	models/experiments/challenge/final/1/data_fill_mean_means.pickle
	models/experiments/challenge/final/1/lda.pickle

Error: "./next.sh p000142" failed (status 1) after 1.84e+09 instructions

---- output of 'next.sh p000142' ----
Traceback (most recent call last):
  File "./get_sepsis_score.py", line 123, in <module>
    pipeline = generate_pipelines()
  File "./get_sepsis_score.py", line 33, in generate_pipelines
    data_means = load_pickle(directory + '/data_fill_mean_means.pickle')
  File "/home/vmuser/omni.py", line 9, in load_pickle
    obj = pickle.load(file)
ImportError: No module named 'pandas.core.internals.managers'; 'pandas.core.internals' is not a package

How am I supposed to load my model parameters? The xgb and lda files are saved classifiers and the other 3 are saved numpy arrays, I have no idea how to load given that I arrive at this error. 

Stack overflow tells me to pip update my pandas version, but it seems no one has had any luck using pip.

Best,
James

[Clare Bates Congdon]

Hi James,

FWIW, I used pickle as well, and had no trouble.

I notice that the first section of your output is a warning, not an
error. Looking at the errors section, it seems that you're being
stopped by the missing panda.core.internals... I guess you know that,
from the Stack Overflow post. (There might also be sort of a mismatch
between the pickled version and the version running on the servers
that's causing the problem.... it might be that if you're able to
redesign your code to not use that module, you'd be fine.)

I'm not clear on how others resolved version mismatches or set the
version on the server.

Best,
Clare

> --
> You received this message because you are subscribed to the Google Groups "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to physionet-challe...@googlegroups.com.
> Visit this group at https://groups.google.com/group/physionet-challenges.
> To view this discussion on the web visit https://groups.google.com/d/msgid/physionet-challenges/b3d633a6-ca3b-4eae-93ca-37b5a1018ea3%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Missagh Dayer]

Hi James

Of the scenarios Clare has pointed out, the version mismatch seems the more likely one to me. I would suggest declaring the specific versions in your dependencies file.

Taking a quick look at the official depositories of Physionet, I saw some outdated versions. They are trying to provide the newer versions for us but there are bound to be problems. (Which is why I decided to use the less version-dependent environment of matlab for my code instead)

To view this discussion on the web visit https://groups.google.com/d/msgid/physionet-challenges/CAD_UTzc-m-Wz_TZwFb9HA9CSr%2BhcvRcx6SRjdNChQ7gky4D5JQ%40mail.gmail.com.