NaNs detected - power overflow?

[Justin Wang]

Hello TVB Users and Experts!

I am trying to replicate the Virtual Aging Brain showcase tutorial (https://lab.ch.ebrains.eu/user-redirect/lab/tree/shared/SGA3%20D1.2%20Showcase%201/virtual_ageing and https://github.com/ins-amu/virtual_aging_brain) using another dataset.

Here is my modelling code:

https://github.com/justinliuyuwang/tvb_cedar/blob/main/model.py

I am performing a parameter sweep and I receive the following warnings with higher noise values. In this case, my parameters are my_magic=1, my_noise=0.04, my_G= 2.2098291647:

WARNING  NaNs detected...
WARNING  NaNs detected...
WARNING  NaNs detected...
WARNING  NaNs detected...
...
WARNING  NaNs detected...
WARNING  NaNs detected...
WARNING  NaNs detected...
WARNING  NaNs detected...
/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py:283: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  return numpy.array([k1, k2, k3])
/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py:334: RuntimeWarning: overflow encountered in power
  dv = (1. / self.tau_o) * (f - v ** (1. / self.alpha))
/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py:336: RuntimeWarning: overflow encountered in power
  (v ** (1. / self.alpha)) * (q / v))
/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py:334: RuntimeWarning: invalid value encountered in power
  dv = (1. / self.tau_o) * (f - v ** (1. / self.alpha))
/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py:336: RuntimeWarning: invalid value encountered in power
  (v ** (1. / self.alpha)) * (q / v))
/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py:336: RuntimeWarning: invalid value encountered in true_divide
  (v ** (1. / self.alpha)) * (q / v))
--- Logging error ---
Traceback (most recent call last):
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/logging/handlers.py", line 70, in emit
    self.doRollover()
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/logging/handlers.py", line 393, in doRollover
    os.remove(dfn)
FileNotFoundError: [Errno 2] No such file or directory: '/home/jwangbay/.tvb-temp/logs/library.log.2023-08-06'
Call stack:
  File "/home/jwangbay/scratch/tvb/paramsearch/scripts/single_param_sim.py", line 41, in <module>
    p.start()
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/multiprocessing/process.py", line 112, in start
    self._popen = self._Popen(self)
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
    code = process_obj._bootstrap()
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/jwangbay/scratch/tvb/paramsearch/scripts/single_param_sim.py", line 10, in func
    output=model1.process_sub(magic,noise,current_G,subject,FCD_file)
  File "/scratch/jwangbay/tvb/paramsearch/scripts/model1.py", line 88, in process_sub
    bold_t, bold_d = utils.tavg_to_bold(tavg_t, tavg_d, tavg_period=1.)
  File "/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py", line 12, in tavg_to_bold
    bold_data_analyzer  = bold_model.evaluate()
  File "/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py", line 222, in evaluate
    self.log.warning("NaNs detected...")
Message: 'NaNs detected...'
Arguments: ()
/scratch/jwangbay/tvb/virtual_ageing/showcase1_ageing/simulation.py:238: RuntimeWarning: invalid value encountered in true_divide
  y_bold = numpy.array(self.V0 * (k1 * (1. - q) + k2 * (1. - q / v) + k3 * (1. - v)))
/scratch/jwangbay/tvb/virtual_aging_brain/env/lib/python3.7/site-packages/numpy/lib/function_base.py:2642: RuntimeWarning: invalid value encountered in true_divide
  c /= stddev[:, None]
/scratch/jwangbay/tvb/virtual_aging_brain/env/lib/python3.7/site-packages/numpy/lib/function_base.py:2643: RuntimeWarning: invalid value encountered in true_divide
  c /= stddev[None, :]

I suspect the following lines of code in simulation.py are giving the warning.

        for step in range(1, t_int.shape[0]):
            state[step, :] = scheme(state[step - 1, :], self.balloon_dfun,
                                    neural_activity[step, :], local_coupling, stimulus)
            if numpy.isnan(state[step, :]).any():
                self.log.warning("NaNs detected1...")

Looking at the logs, the NaNs may be coming from the overflow in the power and divide functions used here in the Balloon model. Has anyone come across a similar issue? There are no NaNs in my input structural connectivity matrix and the simulation runs fine with the same input matrix at lower noise values.

Thank you for your help,

Justin Wang

[WOODMAN Michael]

hi

is your inptu to the bold model the r variable always positive and not too large ? 

the balloon windkessel model can have significant loss of numerical precision so sometimes it's important to rescale the input prior to running it. 

cheers

marmaduke

---

From: tvb-...@googlegroups.com <tvb-...@googlegroups.com> on behalf of Justin Wang <wangliu...@gmail.com>
Sent: Tuesday, August 29, 2023 11:42:20 PM
To: TVB Users
Subject: [RESEAUX SOCIAUX] [TVB] NaNs detected - power overflow?

 

--
You received this message because you are subscribed to the Google Groups "TVB Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tvb-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tvb-users/3ed6b5fa-756d-406a-91e1-779c934e67e8n%40googlegroups.com.

[Justin Wang]

Thank you so much for the prompt and helpful response!

It does indeed look like we have some extremely high values for our firing rate variable, r . Would you happen to have any suggestions or go-to approaches for constraining r during simulations? It seems that when the firing rate spikes, it can cause r to eventually flatline. For this reason, post-hoc value constraints are probably not optimal for us.

Attached: a example time series with a relatively smaller spike in r, but still featuring a flatline later in the simulation.

Thank you again!

Justin

[Justin Wang]

Thank you for your help! This was resolved by halving dt in the integrator until the NaNs disappeared. 

In case anyone is having the same issue, see documentation for dt here: https://docs.thevirtualbrain.org/api/tvb.simulator.integrators.html#id5