Troubleshooting serialization failures & speeding up compile times

[Athena]

So I'm trying to serialize a boost::filesystem::path. So I have:

ar(CEREAL_NVP(Filename));

that I use to serialize such a variable.
Then, inside an included header, I have

namespace cereal
{
    std::string save_minimal(const cereal::XMLOutputArchive&, const boost::filesystem::path& Path);
}

Now, if I have this code, then cereal complains to me "cereal found more than one compatible output serialization function for the provided type and archive combination."
So to troubleshoot, I tried commenting out that function. Then cereal tells me "cereal could not find any output serialization functions for the provided type and archive combination."
This seems weird to me. The above is the only function that can serialize a boost::filesystem::path that I can see in the project (and yes, it's a minimal project to aid troubleshooting).

Maybe I'm doing something wrong here.
Also, regarding the above, for compile time's sake (seriously, compile times with cereal is ridiculously long), I tend to forward declare all serialization functions and define them in separate source files to cut down compile time. I'm not sure if this is the right way or not because the doc doesn't mention it at all. I typically do

namespace cereal
{
    std::string save_minimal(const cereal::XMLOutputArchive&, const boost::filesystem::path& Path);
    void load_minimal(const cereal::XMLInputArchive&, boost::filesystem::path& Path, const std::string& val);
}

Is this the right way of doing it? Should I be doing it in another way?

[Athena]

Alright, so putting the functions in the boost::filesystem namespace works. Still not sure why. The doc says that it's fine to put them in the cereal namespace.

[w.shan...@gmail.com]

Well it was a little tricky but I know what's wrong here:

When cereal performs a save_minimal returning an std::string, it ends up invoking the archive on the string, which is basically the same as if you had serialized a string to begin with.  boost::filesystem::path can be implicitly constructed from an std::string, so string sees two viable overloads for serialization.  Your serialization code for the path is working 'correctly' and sees only one valid overload.

The reason you don't see a problem when you place the code into the boost::filesystem namespace is that ADL can't see that when serializing an std::string, which can only see the std and cereal namespaces.

You might have noticed that using a vanilla serialization function works in this situation.  The reason for that is the argument type for boost::filesystem::path is passed by reference, which will not allow an implicit conversion, whereas the const & used by the split serialization functions does indeed allow implicit conversion.

So your workaround is the best solution for this right now unless I or someone else can think up a better way to detect this.  We do a lot of checking for the minimal serialization, mainly revolving around this very issue (implicit conversion to const &), but it is not without problems (see https://github.com/USCiLab/cereal/issues/132).  We don't use any of these advanced checks for the other serialization functions at the moment.

As far as compilation being slow, I can't say I've really noticed the same, though we do have one post on the list regarding compilation time for polymorphic support.  If you are explicitly instantiating your template parameters than your method is the best speed up you will get.

[Athena]

Ah, I see. Sounds like this is a pain to detect all these things.

As for compile times, I've constantly noticed that serializing pretty simple things still can result in several seconds compile time. This is probably from all the template instantiations (precompiled headers do not help).
I'm not sure if it help in this case, but have you considered a for each argument approach instead of recursion? Here's a video that explains how to achieve for each argument: https://www.youtube.com/embed/Za92Tz_g0zQ?vq=highres&autoplay=0

I did some tests on it with GCC and MSVC. I just created two functions, one which does recursion and one which uses the for each argument method. I pushed the compilers to the limit by passing like 500-600 the variadic template function for MSVC and 800 for GCC. It turns out that there was a difference about 10X in compile speed between the two approaches. That's a lot! Furthermore, the recursion ran out of stack space after about 400 invocations. Ouch. Lastly, MSVC completely failed at optimizing the recursive approach. GCC managed to inline every call just fine, though, resulting in a ridiculous speed boost for GCC vs MSVC. Both MSVC and GCC managed to inline every invocation of the for each argument approach, though. Something to think about.

[w.shan...@gmail.com]

Will definitely take a look at that, as I actually have a bug report that crops up due to recursion blowing out the stack (https://github.com/USCiLab/cereal/issues/185).