voila view indexing

[Thomas Conway]

Hi Majiqians,

Thanks for your work in developing this platform! I'm just trying it out, and so far it's pretty promising.

I'm noticing that the indexing phase when `voila view` is starting up takes quite a long time. Is there a way I can pre-build the index (e.g. running on our HPC, where I am running the build and psi phases), before invoking view (which for network port reasons, I do on my local machine not the HPC)?

Thanks,

Tom.

[San Jewell]

Hi Tom,

Yes, you can pre-build the index. You can use switches --index-file to specify a specific hdf5 file to store the index / use the index from. (and then move that file locally to use) Additionally, specifying more cores with -j <n> will make the indexing run quicker. Finally, most of the time taken in building the index is determining lsv types (3', cassette, mxe, etc) for the checkboxes on the landing page. If you don't require this kind of filtering you can speed it up significantly by specifying --skip-type-indexing.

Let me know if it helps, 

-San

[Thomas Conway]

Just to clarify, can you clarify what command line I should use to just precompute the index and exit?

[San Jewell]

Hi Tom,

There isn't a  command to specifically build the index and exit at this time. It would look something like this:

After the line "Writing index" the index file is created, the process can be exited and the index file can be copied and used elsewhere.

If you are concerned about this writing with a grid/queue manager system, I'd recommend running in an interactive session so that you can check when to exit. I'll consider adding the index step as a stand alone step in future releases.

Thanks,

-San

[Thomas Conway]

Thanks San,

Given the way our environment works, it will be very preferable to build the index in the pipeline phase of our system, so it would be a good addition for us.

[Thomas Conway]

FWIW, a short term work around is to specify a privileged port, so that `voila view` fails when it gets to that point. Ugly, but it enables us to script the index creation.