Merging virtual datasets

[Adrian Fraser]

Hi all,

I'm running into filenumber quotas on Pleiades, and it would be great if I could merge datasets and have a single .h5 file rather than a virtual file and all the, e.g., slices_s1/slices_s1_p*.h5 files. It looks like this should be doable with post.merge_virtual_analysis(), but I'm getting errors along the lines of:

KeyError: "Unable to synchronously open attribute (can't locate attribute: 'set_number')"

and

KeyError: "Unable to synchronously open attribute (can't locate attribute: 'file_number')"

whenever I try. Is there some other method I should be using?

Thanks,

Adrian

[Adrian Fraser]

Here's the complete error message in case anyone has a chance to take a look.

[Daniel Lecoanet]

Hi Adrian,

You might want to look at this pull request. If you are especially keen you could try to implement the proposed changes so that we could merge a fix in (at this stage probably more efficient to have a new PR because this one would require significant rebasing).

Daniel

-- 
You received this message because you are subscribed to the Google Groups "Dedalus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dedalus-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dedalus-users/5215fdc7-4c1e-43b6-b8a5-38f51a56fa75n%40googlegroups.com.
<merge_error.txt>

[Calum Skene]

Hi Adrian and Daniel,

I also would like this so I can cut down on the number of files I have.

I had a quick go here

https://github.com/csskene/dedalus/tree/fix_merge_virtual

it seems to work for the Rayleigh-Benard example (created virtual, merged them and then plotted snapshots from the merged files). Do you want to have a go Adrian and see if it works for you? Probably best to try on example files or copies of your data to start with in case I have overlooked something.

If you think it is working I can open a pull request.

Thanks,

Calum