Bug report

87 views
Skip to first unread message

Yevhenii Prokopalo

unread,
Nov 27, 2018, 6:21:42 AM11/27/18
to beat-devel
Hello, i want to report a bug and ask few questions.
I'm using beat.editor to set up my experiment. Some time during making changes of toolchain they aren't well saved to json file, for exemple:
I deleted the connection betwen two blocs, it was successfuly deleted from list of the connections, but it rest in the dictionary of the connections in the end of file. There is a same problem with changing names of input output fields. In this case web server show that tollchain and experiment is valid, but attempt of starting experiment with beat exp run returns an error. Manual modification of json file resolve this problem.

Also i have a question. I'm trying to run an NN training experiment on your platform. My data preparetion block should return me batches for training. In the training block i want to save all batches and then run few epochs of training using the batches in the random order.
In your iris exemple you save all data to dictionary and then use it to train the model. In my case i can't do the same, because of big amaunt of training data, so i find out another solution: i want to save batches on the disc and keep in memory only the links on the batches. Then i figured that all data wich send between  blocs saved in cache so it should be possibility use it. I didn't find out solution here: https://www.idiap.ch/software/beat/docs/beat/docs/master/beat.web/doc/user/algorithms/guide.html
For my task i need to create All-to-one synchronisation. Is there a possibility to do this?

Zohreh Mostaani

unread,
Nov 27, 2018, 11:15:16 AM11/27/18
to beat-devel
Hello,

About the first problem, it used to be an issue but it has been solved. Please update your beat packages and try again.

About the second problem, you cannot save any data on the disc and even though the data are stored in the cache they are not accessible to users because of security issues. However the new released platform has a new type of algorithm named "Autonomous" that is I think what you need. In this type of algorithm, all the data that has been outputted from the previous block is accessible to the new block immediately. We are updating the documentation for these cases. I suggest you check the "autonomous algorithm" and "Data loader" in the following documentation:

Best,
Zohreh

Yevhenii Prokopalo

unread,
Nov 29, 2018, 8:58:10 AM11/29/18
to beat-devel
Hello, is API v2 available for the local version of the platform? I tried to reinstall beat.editor with conda like it shown here: 
but it returns an error:


so i added an another chanel like i did before:

conda config --env --add channels https://www.idiap.ch/software/beat/conda/label/beta

it solved the problem, but the version of the editor is the same.

Zohreh Mostaani

unread,
Nov 29, 2018, 11:05:08 AM11/29/18
to beat-...@googlegroups.com
Hello,

You did not need to reinstall everything. You needed to update all the conda packages after activating your conda environment:

"conda update beat.editor"
"conda update beat.backend.python"
and so on for all the "beat" packages. However if you installed everything from scratch you should have the latest version.

Do you still have the same problem with the toolchain?

The local beat supports api version 2. Beat.editor is not still able to validate the new algorithms so you may see "invalid" in front of those algorithms in the list of your algorithms but the experiments should run without any problem.

Best,
Zohreh

--
You received this message because you are subscribed to the Google Groups "beat-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beat-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Evgeniy Prokopalo

unread,
Nov 29, 2018, 11:21:16 AM11/29/18
to beat-...@googlegroups.com
Hello, 
Thank you,
How can i set algorithm type? Manually change the json file?


чт, 29 нояб. 2018 г. в 17:05, Zohreh Mostaani <mostaan...@gmail.com>:

Zohreh Mostaani

unread,
Nov 30, 2018, 5:23:49 AM11/30/18
to beat-devel
Hello,

It is not only about the json file but also about the python file. What I suggest to you at the moment, if you are using beat.editor locally to debug your system, is to make your algorithm on the platform (it does not necessarily be completely correct) and then pull it locally using "beat pull" commands. like this you can be sure that the templates for both python and json files are correct. Then you can go on as normal and debug your algorithm.

If you have not configured your local system to be able to pull from the platform you can look at this documentation:


Best,
Zohreh

Yevhenii Prokopalo

unread,
Dec 3, 2018, 4:39:13 AM12/3/18
to beat-devel
Hello,

I have one more question, in api v2 during the forwarding data between blocks, data is saved in RAM or on the disk? Because when i was talking about big amount of data i meant hundreds gigas.

Best,
Yevhenii

SG

unread,
Dec 3, 2018, 5:50:07 AM12/3/18
to beat-devel
Hi,

Currently, the data will be loaded from disk.

Can you give more precise numbers on the volume of data you expect to generate as well as load in memory ?

Best regards
SG.
Reply all
Reply to author
Forward
0 new messages