Recapitate a SLiM 3.6 tree into pyslim 0.700 with msprime.Demography(): metadata issue?

[Chrystelle Delord]

Dear everyone,

I could not figure out if this was more of a SLiM, or pyslim issue, but since the problem seemed to be related to SLiM metadata, I end up posting here!

I did not have any trouble to recapitate a tree generated from SLiM ver. 3.6 when using the legacy function orig_ts.recapitate() from pyslim ver. 0.700. However, when I wanted to switch to the new function pyslim.recapitate(orig_ts, demography=demography, ...) with a msprime.Demography() object, I got an error message starting with:

"MetadataValidationError: 'slim_id' is a required property

Failed validating 'required' in schema:"

(and then there was a large print of OrderedDict{} for each subpopulation, in I understood well).

I feel like this could just be an issue of SLiM version, when reading the section 26.4.4 from the SLiM Manual from Feb.2022 ? Are the trees metadata read the same way as it used to be, by the new pyslim.recapitate(ts, ...) function compared to the  ? Before I re-run all my Eidos scripts with SLiM 3.7, I just wanted to make sure the problem could not arise from anything else.

My SLiM and pyslim scripts are here enclosed.

Thank you so much for your help, have a great week-end!

Chrys

[Ben Haller]

Hi Chrys.  It seems highly likely to be a version mismatch, although I can't promise that.  :->  I would definitely recommend that you update to SLiM 3.7.1.  In general, if you update one part of your toolchain you probably want to update all of them; we do try to preserve backward compatibility where possible, but in general these packages are developed in synchrony.  Perhaps Peter will have more to say about this, but that's my take.

Cheers,
-B.

Benjamin C. Haller
Messer Lab
Cornell University

[Chrystelle Delord]

Thanks a lot for your very quick answer, Ben!

I had just saw the details from Peter after the announce of SLiM3.7 release (https://groups.google.com/g/slim-discuss/c/5KXQC3G1Dbk).

I will follow your advice and jump to 3.7 as it will be better anyway,

Cheers!

Chrys

[Peter Ralph]

Well, Ben's right that updating SLiM should fix this. pyslim is meant
to be backwards compatible; it looks like what's happening is that
pyslim should be catching that this is an older tree sequence and
updating it before proceeding with recapitation. I've opened an issue:
https://github.com/tskit-dev/pyslim/issues/250

thanks!

> --
> SLiM forward genetic simulation: http://messerlab.org/slim/
> ---
> You received this message because you are subscribed to the Google Groups "slim-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to slim-discuss...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/slim-discuss/8978a00d-490e-43bc-8f15-e71146aae91fn%40googlegroups.com.

[Chrystelle Delord]

Hi and thank you both!

Unfortunately, it turns out that upgrading to SLiM 3.7 did not solve the issue.

I also tried doing rts = pyslim.recapitate(pyslim.SlimTreeSequence(ts), demography=demography, recombination_rate=rate_map, random_seed=1), as suggested in the issue opened by Peter, but that did not work either. Recapitation without a msprime.Demography object still works, so maybe should I try another solution to simulate past demographic events?

I just needed to simulate a past divergence event at a specified time ago t_split. Is that possible using the legacy function ts.recapitate() ?

Many thanks!

Best regards,

Chrystelle

[Chrystelle Delord]

Ok I found my mistake!!

When defining my msprime.Demography object(), I used add_population to add an ancestral population called "pop_anc".

Apparently this was wrong, because that population did not exist in the initial tree from SLiM, which is why it appeared nowhere in the tree metadata leading to the error message.

So, instead, I used the empty population "pop_0" that already existed in the tree from SLiM to become my ancestral population, just like someone did in this thread: https://groups.google.com/g/slim-discuss/c/GxjnTQZb7RA. This gives the following:

ts = pyslim.load("test.trees")

Ne_demo = 500
t_split = 4*Ne_demo

demography = msprime.Demography.from_tree_sequence(ts)
for pop in demography.populations:
    pop.initial_size = Ne_demo
# demography.add_population(name="pop_anc", initial_size= int(Ne_demo*2))  # This is the line to be removed.
demography.add_population_split(time=t_split, derived=["p1", "p2"], ancestral="pop_0")  # Use pop_0 instead, it already exists in the tree.
demography.set_symmetric_migration_rate(populations=["p1", "p2"], rate=0.05) 

rts = pyslim.recapitate(ts, demography=demography, recombination_rate=rate_map, random_seed=1) # Now works!

So this is great, and I guess it is actually possible to add a new, unexisting population to the tree, but then we have to set its metadata manually?

Sorry for the confusion!

Thank you so much again for all the help you provide to us users, with the endless SLiM/pyslim possibilities!!

Best regards,

Chrystelle

[Ben Haller]

Hi Chrystelle!  I'm glad you found the problem; thanks for posting the solution to the list.  Happy modeling!

Cheers,
-B.

Benjamin C. Haller
Messer Lab
Cornell University

[Peter Ralph]

Oh, good! I'm glad you got this - and, apologies, I'd meant to track
down the problem but it fell off the top of my inbox.

For future reference, you can indeed add the extra metadata to new
populations (so that your initial method would have worked) - for one
way to do this, see how it works under the hood in `recapitate`:
https://github.com/tskit-dev/pyslim/blob/3033d53b98a6fd51fd3f7c616f61fedd0fc52a8c/pyslim/methods.py#L53

nice work,
Peter

On Tue, May 10, 2022 at 2:46 AM Chrystelle Delord

> To view this discussion on the web visit https://groups.google.com/d/msgid/slim-discuss/eb809235-bb10-4fb7-a6bf-908ce0fb5e17n%40googlegroups.com.

[Chrystelle Delord]

Thank you very much Peter! I am looking forward to using this solution in the future.

I am very sorry to bother you again, I just have one related question if this is ok?

If I simulate a demographic event in SLiM (for instance a sudden change in size from N=5000 to N=1000 in each subpopulation, forward in time), and then use pyslim to recapitate, I will have to set pop.initial_size to 1000 in the msprime.Demography() object.

But, is it needed to re-model the population change in size in msprime.Demography() (so this time, a change from N=1000 to N=5000, backward-in-time)?

Or will the initial_state as a SLiM tree, already provide all the information through its topology (and then, the only demographic events to setup in msprime.Demography() will be those more ancient than the first generation simulated in SLiM)?

Have a great day everyone,

Best regards

Chrystelle