Working with const buffers (using a different ExpandEmitter)

[Darren Garvey]

Hi,

I had an idea that the ctemplate Template::Expand function could be made more efficient by avoiding copying the template/dictionary data where possible. I haven't succeeded so I was wondering what I am doing wrong.

----background
The library I'm messing with uses `boost::asio::const_buffer` when sending data, a const_buffer being a void* into the data and an indication of its length. The const_buffer doesn't own the underlying memory, so that has to remain valid. A vector<const_buffer> can be output directly, taking advantage of scatter-gather I/O when available.
----background--

The attached (small) patch does need a prerelease version of Boost, but it might be obvious from looking what's wrong. Not only is it not any faster than expanding to a std::string - at least not with a small-to-medium-sized template - but it doesn't even get the dictionary tokens properly. The Emit(const char*) and Emit(const char*, size_t) functions seem to the only ones called when modifiers aren't used, but do they actually point to the memory of the template/dictionary values? They don't point to temporaries do they?

I'm a bit stuck.

Regards,
Darren

P.S. Because the const_buffer doesn't own the memory, a string is passed to Expand to pick up individual chars and const std::string&s (which I figure refer to temporaries).

[Craig Silverstein]

} I had an idea that the ctemplate Template::Expand function could be
} made more efficient by avoiding copying the template/dictionary data
} where possible.

An interesting idea! Once it's working, I'll be interested to see
what kind of performance behaviors you have. Even without using
boost, it would be possible to add this to ctemplate, using a vector
of TemplateString's (which are much like the boost class you're
using), and a helper routine that we'd add to Template that emits such
a vector to an fd using writev (which is, I assume, what the boost
routine is doing as well).

As for variable lifetime, the Emitter class was written with no
assumptions for the input lifetimes, but we could do an audit to
figure out more exactly what we can depend on.

In general, literal text is owned by the Template instance and should
live as long as you could ever need. Variables are owned by the
associated TemplateDictionary, which should also be long enough for a
given Expand() call. Modified variables, as you point out, do not
live for very long. Neither does various debug info that might be
emitted into the expanded template, none of which should apply to your
test cases.

I think to further debug this, we'd have to see a sample program
you've written that uses this new functionality, and the output it
gives in practice.

craig