Saving and loading in pyMKS

[Andrew Medford]

Is there a standard way to save/load models in pyMKS?

In particular, I want to train a localization model to a large number of data sets, then dump the coefficients somewhere, and re-load them later. I can't find any functionality like this, but wanted to double check before I started implementing something. If I do implement something, are there any I/O guidelines? (specific data structures, data storage format, etc.)

[Daniel Wheeler]

Hi Andrew,

I think that Scikit-learn simply recommends pickling models for persistent use.

http://scikit-learn.org/stable/modules/model_persistence.html

I don't think (I don't know for sure) that there are helper functions
in PyMKS to do this, but it is just a few lines of code assuming that
the PyMKS model instances are picklable. There are always persistence
issues with pickled code and data so maybe store the pickled model
with some meta data about the PyMKS version. Clearly though, this is a
core issue for PyMKS and I think that pickling is not a great long
term solution.

Cheers,

Daniel

> --
> You received this message because you are subscribed to the Google Groups
> "PyMKS General" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pymks-genera...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pymks-general/e6fe9141-03dc-415e-bd64-a99d923c11e4%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Daniel Wheeler

[Andrew Medford]

Thanks Daniel... I will use model pickling in the short term.

As you suggest, the long-term stability of pickles (particularly of pickled high-level objects) is a concern. We probably want to use something like JSON or XML in the longer term, and could come up with standards of what exactly should be saved (e.g. influence coefficients for a MKSLocalizationModel).

David - let me know if you have any thoughts here. I can potentially work on this next year since I will probably need this feature soon.

Best,

AJM

[David Brough]

Andrew,

As you said in the short term pickle will work. I haven't had a lot of time to investigate

this, but there has been some work done to use Predictive Model Markup Language (PMML)

for model persistence.

https://github.com/alex-pirozhenko/sklearn-pmml

Here is some discussion about model persistence on sklearn's mailing list.

http://sourceforge.net/p/scikit-learn/mailman/scikit-learn-general/thread/8737zg1br2.fsf%40berkeley.edu/#msg34386095

Thanks,

David

Hi

To view this discussion on the web visit https://groups.google.com/d/msgid/pymks-general/10891419-a5b0-432e-b976-cbd873f166b1%40googlegroups.com.