Loading Theta matrix after save

[Алексей Астафьев]

Hello,

I've saved all matrices after fitting (p_wt, n_wt).

Than I loaded them from binaries. And:

• model.get_phi() works well
• model.get_theta gives me empty result. Furthermore, model.get_theta_sparse() gives me this exception
   InternalError :  theta_matrix->topic_indices_size() == 0

what should I do to load theta matrix correctly?

[great-mel]

Hi,

you've saved p_wt and n_wt, not theta, so after loading you won't be able to use theta, it's empty. To store and load theta set 'theta_name' parameter in ARTM constructor and deal with theta matrix in the same way, as with pwt and nwt (using get_phi, import/export, attach etc)

Regards, Murat

вторник, 27 июня 2017 г., 17:16:39 UTC+3 пользователь Алексей Астафьев написал:

[Алексей Астафьев]

ok. thanks

[Aleksey Astafiev]

Well I've done this

  model_artm = artm.ARTM(num_topics=T, topic_names=topic_names, class_ids={'@text':1},
                       cache_theta=True, theta_name='theta', seed=1,
                       show_progress_bars=True, num_processors=3)

fitted...

than

  model_artm.save(filename=SAVED_P_WT, model_name='p_wt')
  model_artm.save(filename=SAVED_N_WT, model_name='n_wt')
  model_artm.save(filename=SAVED_THETA, model_name='theta')

after restarting kernel and

  model_artm.load(filename=SAVED_P_WT, model_name='p_wt')
  model_artm.load(filename=SAVED_N_WT, model_name='n_wt')
  model_artm.load(filename=SAVED_THETA, model_name='theta')

then

  theta = model_artm.get_theta()

And still empty theta.

Where am I wrong?

What model_name should be in load method to load it correctly?

On Tuesday, June 27, 2017 at 5:40:14 PM UTC+3, great-mel wrote:

[Мурат Апишев]

Hi, Alexey!

 

Please see the page http://docs.bigartm.org/en/stable/tutorials/python_userguide/different.html from docs, you need paragraph 'Saving/loading model'. Briefly you can store your model completly to disk and than load it back with all matrices.

 

Explicit loading of standart theta matrix via load method is impossible.

 

--

 

Regads, Murat

 

25.01.2019, 15:32, "Aleksey Astafiev" <aast...@gmail.com>:

--
You received this message because you are subscribed to the Google Groups "bigartm-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigartm-user...@googlegroups.com.
To post to this group, send email to bigart...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigartm-users/e7f777ea-2a02-409f-9c7a-3a21f70ad092%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Алексей Астафьев]

Thank you for the answer. It works the same way.

My code:

model_artm = artm.ARTM(num_topics=T, topic_names=topic_names, class_ids={'@text':1},
                       cache_theta=True, theta_name='theta', seed=1,

                       show_progress_bars=True, num_processors=7)

model_artm.initialize(local_dictionary)

fitting...

than:

model_artm.dump_artm_model(MODEL)

In MODEL directory I've got:

• n_wt.bin
• p_td.bin
• p_wt.bin
• parameters.bin
• parameters.json
• score_tracker.bin

than:

model_artm = artm.ARTM(num_topics=T, topic_names=topic_names, class_ids={'@text':1},
                       cache_theta=True, theta_name='theta', seed=1,

                       show_progress_bars=True, num_processors=7)

model_artm.initialize(local_dictionary)

model = artm.load_artm_model(MODEL)

than:

theta_data = model_artm.get_theta_sparse()

Output:

---------------------------------------------------------------------------
InternalError                             Traceback (most recent call last)
<ipython-input-16-3776f41e3084> in <module>
----> 1 theta_data = model_artm.get_theta_sparse()

/usr/local/lib/python3.6/site-packages/bigartm-0.9.0-py3.6.egg/artm/artm_model.py in get_theta_sparse(self, topic_names, eps)
    886             for topic_name in topic_names:
    887                 args.topic_name.append(topic_name)
--> 888         theta = self._lib.ArtmRequestThetaMatrixExternal(self.master.master_id, args)
    889 
    890         numpy_ndarray = numpy.zeros(shape=(3 * theta.num_values, 1), dtype=numpy.float32)

/usr/local/lib/python3.6/site-packages/bigartm-0.9.0-py3.6.egg/artm/wrapper/api.py in artm_api_call(*args)
    159                 func.restype = spec.result_type
    160             result = func(*c_args)
--> 161             self._check_error(result)
    162 
    163             # return result value

/usr/local/lib/python3.6/site-packages/bigartm-0.9.0-py3.6.egg/artm/wrapper/api.py in _check_error(self, error_code)
     95             exception_class = ARTM_EXCEPTION_BY_CODE.get(error_code)
     96             if exception_class is not None:
---> 97                 raise exception_class(error_message)
     98             else:
     99                 raise RuntimeError(error_message)

InternalError: theta_matrix->topic_indices_size() == 0

[Алексей Астафьев]

Sorry, I've found the error! It works fine now.

It should be:

model_artm = artm.load_artm_model(MODEL)

copy paste damn