m = hddm.HDDM(data, ...)
m.mcmc(dbname='test.db')
m.sample(5000)
# later point in time
m = hddm.HDDM(data, ...) # reconstruct model
m.load_from_db(dbname='test.db') # load traces
Hi Thomas,
Oystein is working with me (Guido).
Here is why we want to save the complete model, or its definition and the traces separately:
We have access to a grid/cluster where it is best to submit different chains as independent jobs (which are submitted by slurm to independent machines ). If we later want to test the convergence of these chains, we need the traces if we want to use the gelman-rubin convergence criterion (which Øystein found implemented in the R_hat function of the kabuki module).* the information about the model should then be helpful to know which trace is associated with which parameter. hence, we need to keep track of model definition and save traces. It is not a big issue to slighlty re-organize our procedure so that the model definition is saved in a variable or just as a string in a text file.
I would have one question though: What is with the information about number of burn in samples? I would think that within chain variance should only be computed on samples after completion of the burn in. Is this what your R_hat function does and does the data-base which saves the samples also save information about the number of burn in samples in a way that this would be taken into account by the R_hat function?
--------------------------------------------------------------------------- DatabaseError Traceback (most recent call last) <ipython-input-25-d96cf8e228e0> in <module>() ----> 1 m_within_subj.mcmc(dbname='traces.db', db='sqlite') /Users/benjamin/anaconda/lib/python2.7/site-packages/kabuki/hierarchical.pyc in mcmc(self, assign_step_methods, *args, **kwargs) 539 """ 540 --> 541 self.mc = pm.MCMC(self.nodes_db.node.values, *args, **kwargs) 542 543 self.pre_sample() /Users/benjamin/anaconda/lib/python2.7/site-packages/pymc/MCMC.pyc in __init__(self, input, db, name, calc_deviance, **kwds) 80 name, 81 calc_deviance=calc_deviance, ---> 82 **kwds) 83 84 self._sm_assigned = False /Users/benjamin/anaconda/lib/python2.7/site-packages/pymc/Model.pyc in __init__(self, input, db, name, reinit_model, calc_deviance, verbose, **kwds) 205 # Specify database backend and save its keywords 206 self._db_args = kwds --> 207 self._assign_database_backend(db) 208 209 # Flag for model state /Users/benjamin/anaconda/lib/python2.7/site-packages/pymc/Model.pyc in _assign_database_backend(self, db) 572 self._db_args['dbname'] = self.__name__ + '.' + db 573 --> 574 self.db = module.Database(**self._db_args) 575 elif db in database.__modules__: 576 raise ImportError( /Users/benjamin/anaconda/lib/python2.7/site-packages/pymc/database/sqlite.pyc in __init__(self, dbname, dbmode) 187 self.cur = self.DB.cursor() 188 --> 189 existing_tables = get_table_list(self.cur) 190 if existing_tables: 191 # Get number of existing chains /Users/benjamin/anaconda/lib/python2.7/site-packages/pymc/database/sqlite.pyc in get_table_list(cursor) 264 SELECT name FROM sqlite_master 265 WHERE type='table' AND NOT name='sqlite_sequence' --> 266 ORDER BY name""") 267 return [row[0] for row in cursor.fetchall()] 268DatabaseError: file is encrypted or is not a database
---------------------------------- ERROR MESSAGE WHILE TRYING TO RELOAD DATA FROM PICKLE DB -----------------
--------------------------------------------------------------------------- DatabaseError Traceback (most recent call last) <ipython-input-24-3780245ba32c> in <module>() ----> 1 m_within_subj.load_db('traces.db') /Users/benjamin/anaconda/lib/python2.7/site-packages/kabuki/hierarchical.pyc in load_db(self, dbname, verbose, db) 797 798 # Open database --> 799 db = db_loader(dbname) 800 801 # Create mcmc instance reading from the opened database /Users/benjamin/anaconda/lib/python2.7/site-packages/pymc/database/sqlite.pyc in load(dbname) 236 Return a Database instance. 237 """ --> 238 db = Database(dbname) 239 240 # Get the name of the objects /Users/benjamin/anaconda/lib/python2.7/site-packages/pymc/database/sqlite.pyc in __init__(self, dbname, dbmode) 187 self.cur = self.DB.cursor() 188 --> 189 existing_tables = get_table_list(self.cur) 190 if existing_tables: 191 # Get number of existing chains /Users/benjamin/anaconda/lib/python2.7/site-packages/pymc/database/sqlite.pyc in get_table_list(cursor) 264 SELECT name FROM sqlite_master 265 WHERE type='table' AND NOT name='sqlite_sequence' --> 266 ORDER BY name""") 267 return [row[0] for row in cursor.fetchall()] 268DatabaseError: file is encrypted or is not a database
--
You received this message because you are subscribed to the Google Groups "hddm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hddm-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.