[boost] Pimpl Again?

[Vladimir Batov]

Don't laugh but it is only fairly recently at work that I managed to
move all our supported platforms to C++11. So, then, I started moving
our code to C++11 (what a delight!) and stumbled upon my old pimpl which
is used quite a bit around my workplace for all its good properties...
:-)

Now C++11-ed piml turned out to be very small and very basic. So basic
it felt it did not have the right to exist. Still, it encapsulates and
enforces the "proper" pimpl properties and behavior, reduces
implementation minutia and offers a recognizable deployment pattern
(helps other people reading the code).

Over the years IMO the technique has been proven as legitimate and
useful, a basic but important building block... rather than a curiosity
item. Unfortunately, despite many articles and a few Sutter's GotWs
about it there is nothing ready-to-go like std::unique_ptr.

I feel that pimpl and its deployment pattern need to be codified,
described, "standardized" and in our toolboxes to stop/avoid everyone
re-inventing the pimpl and making the same mistakes. IMO Boost is best
positioned for that.

Do you think we might review what I've got and/or maybe collectively
come up with something better... It's no MPL or Hana but as
std::unique_ptr it's one of the first "little things" I personally reach
out for in my everyday work. IMO having pimpl in Boost would save quite
a few hours of frustration for many.

Thoughts? No pressure. I am easy either way. :-)

https://github.com/yet-another-user/pimpl

Docs are freshened up but still somewhat out of date. Apologies.



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Chris Glover]

>
>
> Thoughts? No pressure. I am easy either way. :-)
>
> https://github.com/yet-another-user/pimpl
>
> Docs are freshened up but still somewhat out of date. Apologies.
>
>
>

I would personally use this, I am sure of that, as I type this in by hand
using unique_ptr all the time.

One addition I would like to see is a version where the heap allocation is
avoided by storing an internal buffer in the base class. Yes, this means
one needs to keep the size in sync between the impl and the header, but
this can be an important optimization sometimes.

-- chris

[Seth]

On 28-05-16 16:25, Chris Glover wrote:
> One addition I would like to see is a version where the heap allocation is
> avoided by storing an internal buffer in the base class. Yes, this means
> one needs to keep the size in sync between the impl and the header, but
> this can be an important optimization sometimes.

Don't forget about alignment requirements.

I agree. Pimpl serves different goals. If the goal is just to hide
implementation details (but not necessarily avoid recompilation on
implementation change) this is very welcome. Though, soon, you'd verge
to a "opaque value_ptr" where the value_ptr could have clone-semantics.

So, I can see a choice for "pure Pimpl" facilities, without any optional
stuff.

Seth

[Vladimir Batov]

On 2016-05-29 00:25, Chris Glover wrote:
>>
>> https://github.com/yet-another-user/pimpl

>>
> I would personally use this, I am sure of that, as I type this in by
> hand
> using unique_ptr all the time.

Glad to hear that... and IMO the proposed design does have advantage
over unique_ptr-based pimpl as IMO unique_ptr hardly has any advantage
over the raw pointer -- even the destructor has to be explicit
non-default and non-inlined. When the proposed design IMO cuts down on
implementation minutia.

> One addition I would like to see is a version where the heap allocation
> is
> avoided by storing an internal buffer in the base class.

I feel that the proposed policy/manager-based design allows us to supply
a manager as per your requirements. I personally have never used such
design but I am certainly eager to see it implemented, tested and
incorporated... if pimpl gets its "foot in the Boost door" so to speak.

[Gavin Lambert]

On 28/05/2016 17:41, Vladimir Batov wrote:
> I feel that pimpl and its deployment pattern need to be codified,
> described, "standardized" and in our toolboxes to stop/avoid everyone
> re-inventing the pimpl and making the same mistakes. IMO Boost is best
> positioned for that.
>
> Do you think we might review what I've got and/or maybe collectively
> come up with something better... It's no MPL or Hana but as
> std::unique_ptr it's one of the first "little things" I personally reach
> out for in my everyday work. IMO having pimpl in Boost would save quite
> a few hours of frustration for many.

I like the idea, certainly. My main concerns about Boost-ification of
this are:

1. The way the docs are structured suggest that the "natural"
implementation is the shared one and the unique implementation is an
extension. Standard C++ language and performance guidelines suggest the
reverse should be preferred (or as Chris suggested, one that avoids heap
allocation entirely). (This is mostly just a doc issue; the actual
implementation seems neutral.)

2. This introduces a symbol (pimpl) into the global namespace, which is
probably against Boost guidelines. But putting it into the boost
namespace doesn't seem like a good solution either as usage requires
explicit template specialisation, which is more clunky if the template
to be specialised is in a different namespace. (Though even being in
the global namespace doesn't avoid this clunkiness, if the user classes
are themselves in a non-global namespace.)

[Gavin Lambert]

On 29/05/2016 02:25, Chris Glover wrote:
> One addition I would like to see is a version where the heap allocation is
> avoided by storing an internal buffer in the base class. Yes, this means
> one needs to keep the size in sync between the impl and the header, but
> this can be an important optimization sometimes.

Ideally the header should store a max size / capacity -- construction
succeeds as long as the "real" impl is equal to or smaller than this.
This allows a bit of flexibility for different layouts used by different
compilers, or for "expansion room" without breaking consumers.

Though you don't want to leave too much wiggle room. Memory is cheap
but cache is less so.

[Vladimir Batov]

On 05/30/2016 10:08 AM, Gavin Lambert wrote:
> On 28/05/2016 17:41, Vladimir Batov wrote:
>> I feel that pimpl and its deployment pattern need to be codified,
>> described, "standardized" and in our toolboxes to stop/avoid everyone
>> re-inventing the pimpl and making the same mistakes. IMO Boost is best
>> positioned for that.
>>
>> Do you think we might review what I've got and/or maybe collectively
>> come up with something better... It's no MPL or Hana but as
>> std::unique_ptr it's one of the first "little things" I personally reach
>> out for in my everyday work. IMO having pimpl in Boost would save quite
>> a few hours of frustration for many.
>
> I like the idea, certainly. My main concerns about Boost-ification of
> this are:
>
> 1. The way the docs are structured suggest that the "natural"
> implementation is the shared one and the unique implementation is an
> extension. Standard C++ language and performance guidelines suggest
> the reverse should be preferred (or as Chris suggested, one that
> avoids heap allocation entirely). (This is mostly just a doc issue;
> the actual implementation seems neutral.)

As you point out the described is "just a doc issue". In that light I do
not quite understand how it can be a "main concern". I'll certainly be
re-working docs when/if the time comes. At this point IMO we should
decide if we want pimpl in Boost and if the suggested design could be
used as the starting point.

> 2. This introduces a symbol (pimpl) into the global namespace, which
> is probably against Boost guidelines.

Again, I feel you are rushing things. Adapting doccs and putting it into
Boost namespace are minor issues to be addressed in due course.

> But putting it into the boost namespace doesn't seem like a good
> solution either as usage requires explicit template specialisation,
> which is more clunky if the template to be specialised is in a
> different namespace.

Indeed. Visual Studio did not have that problem for quite some time. GCC
had that "clunkiness" a few years ago. Now I try with gcc-4.8:

namespace boost
{
template<class user_type> struct pimpl {...};
}

struct Shared : public boost::pimpl<Shared>::shared { ... };

Works fine.

> (Though even being in the global namespace doesn't avoid this
> clunkiness, if the user classes are themselves in a non-global
> namespace.)

That's something I do not understand. I have plenty of classes like:

template<> struct pimpl<chart::panel::leg>::implementation.

[Vladimir Batov]

On 05/30/2016 10:12 AM, Gavin Lambert wrote:
> On 29/05/2016 02:25, Chris Glover wrote:
>> One addition I would like to see is a version where the heap
>> allocation is
>> avoided by storing an internal buffer in the base class. Yes, this means
>> one needs to keep the size in sync between the impl and the header, but
>> this can be an important optimization sometimes.
>
> Ideally the header should store a max size / capacity -- construction
> succeeds as long as the "real" impl is equal to or smaller than this.
> This allows a bit of flexibility for different layouts used by
> different compilers, or for "expansion room" without breaking consumers.
>
> Though you don't want to leave too much wiggle room. Memory is cheap
> but cache is less so.

I am personally very open to that idea. Although I would probably
suggest using custom allocator instead. It can be (almost?) as quick,
easily fit to the current design and would not irritate pimpl purists
:-) ... Having said that I feel that discussing in depth another
pimpl::manager is somewhat premature... although I am hoping we'll bet
to that point. In that light I feel that our experience with
boost::convert was quite positive: 1) reviewed; 2) accepted in
principle; 3) extended, improved by a group of enthusiasts who knew that
their effort won't be wasted/dismissed -- that's important IMO as that's
the only reward we can offer.

[Gavin Lambert]

On 30/05/2016 12:29, Vladimir Batov wrote:
> Again, I feel you are rushing things. Adapting doccs and putting it
> into Boost namespace are minor issues to be addressed in due course.

It would be minor if your entire design wasn't based around template
specialisation. Since it is, however, it becomes a major consideration
since it directly impacts the primary API of the library.

This can't exist inside a "namespace chart { namespace panel {" block.

If you have the convention that these blocks span the file, it means you
either have to put such specialisations at the top or bottom of the file
outside the namespace block (which may break locality) or you have to
close the namespace blocks and reopen them later (which is ugly).

(Imagine the case where you want to define some detail classes that only
exist in the implementation *first*, then the pimpl definition, then the
public interface. The first and last should be in the local namespace
[or child thereof]; the second can't be. And they have to be defined in
that order due to the dependencies.)

Granted implementation files don't *have* to have namespace blocks --
they can use using directives or explicit naming instead; but the latter
is ugly and the former risks accidentally defining global symbols, so
it's generally safer to use namespace blocks instead.

[Vladimir Batov]

On 05/30/2016 11:02 AM, Gavin Lambert wrote:
> On 30/05/2016 12:29, Vladimir Batov wrote:
>> Again, I feel you are rushing things. Adapting doccs and putting it
>> into Boost namespace are minor issues to be addressed in due course.
>
> It would be minor if your entire design wasn't based around template
> specialisation. Since it is, however, it becomes a major
> consideration since it directly impacts the primary API of the library.
>
>> On 05/30/2016 10:08 AM, Gavin Lambert wrote:
>>> (Though even being in the global namespace doesn't avoid this
>>> clunkiness, if the user classes are themselves in a non-global
>>> namespace.)
>>
>> That's something I do not understand. I have plenty of classes like:
>>
>> template<> struct pimpl<chart::panel::leg>::implementation.
>
> This can't exist inside a "namespace chart { namespace panel {" block.

You are right. It can't be inside a "namespace chart { namespace panel
{" block... and I personally never expected it to be. After all that's a
specialization of (assume Boost)

namespace boost
{
template<typename user_type> pimpl { struct implementation; }

}

> If you have the convention that these blocks span the file, it means
> you either have to put such specialisations at the top or bottom of
> the file outside the namespace block (which may break locality) or you
> have to close the namespace blocks and reopen them later (which is ugly).
>
> (Imagine the case where you want to define some detail classes that
> only exist in the implementation *first*, then the pimpl definition,
> then the public interface. The first and last should be in the local
> namespace [or child thereof]; the second can't be. And they have to
> be defined in that order due to the dependencies.)
>
> Granted implementation files don't *have* to have namespace blocks --
> they can use using directives or explicit naming instead; but the
> latter is ugly and the former risks accidentally defining global
> symbols, so it's generally safer to use namespace blocks instead.

Yes, I understand. You have your personal programming style and you feel
the suggested pimpl does not fit that style as well as you'd like it to.
I respect that. But I do not see it as a principal objection... is it a
principal objection?

[Gavin Lambert]

On 30/05/2016 13:20, Vladimir Batov wrote:
> You are right. It can't be inside a "namespace chart { namespace panel
> {" block... and I personally never expected it to be. After all that's a
> specialization of (assume Boost)
>
> namespace boost
> {
> template<typename user_type> pimpl { struct implementation; }
> }

Right. And to users, this is going to feel like defining their types
(at least the private members, possibly even some method
implementations) inside the boost namespace -- and that feeling isn't
entirely wrong.

>> If you have the convention that these blocks span the file, it means
>> you either have to put such specialisations at the top or bottom of
>> the file outside the namespace block (which may break locality) or you
>> have to close the namespace blocks and reopen them later (which is ugly).
>>
>> (Imagine the case where you want to define some detail classes that
>> only exist in the implementation *first*, then the pimpl definition,
>> then the public interface. The first and last should be in the local
>> namespace [or child thereof]; the second can't be. And they have to
>> be defined in that order due to the dependencies.)
>>
>> Granted implementation files don't *have* to have namespace blocks --
>> they can use using directives or explicit naming instead; but the
>> latter is ugly and the former risks accidentally defining global
>> symbols, so it's generally safer to use namespace blocks instead.
>
> Yes, I understand. You have your personal programming style and you feel
> the suggested pimpl does not fit that style as well as you'd like it to.
> I respect that. But I do not see it as a principal objection... is it a
> principal objection?

I don't think it's a personal programming style issue. I think it's a
natural consequence of writing code in the non-global namespace.

I suppose you could consider writing code not in the global namespace as
a personal style choice, but at least for library code (which is the
most likely to want to use pimpl style so they can avoid ABI breakage
with applications) this is the encouraged style. Certainly most (all?)
of STL and Boost is itself not in the global namespace.

So (at least in my opinion), the primary users of Pimpl will probably be
library developers who want to write code in custom namespaces. But
they can't use Pimpl without breaking out of their namespace and writing
some code in the global/boost namespace instead. This is a code
uglification that seems like it should be a principal objection -- if
that doesn't count, what does?

(Some app authors will want to use Pimpl as well of course -- which has
the same issues if the namespaces don't match.)

[Emil Dotchevski]

My 2c:

In foo.h, instead of

class foo
{
class pimpl
pimpl * p_;
public:
foo();
~foo();
void do_something();
};

It's better to do it the C way: simply leave foo incomplete:

struct foo;
foo * create_foo();
void destroy_foo( foo * );
void do_something( foo * );

The above is much improved using shared_ptr:

struct foo;
shared_ptr<foo> create_foo();
void do_something( foo * );

Emil

[Vladimir Batov]

Gavin,

Apologies for top-posting... I feel that you make very ambitious
generalizations about how users will feel and how ugly in your view the
code will be of those who might decide to use the proposed pimpl. I hear
your view and I respect it. I just do not happen to share it. No drama.
we are all different.

So, in your view, should we try and have pimpl in Boost? I think we
should as it'd be useful IMO. I proposed the one I use. I am not married
to it. If you have a design which makes you happy, let's consider it...
Otherwise, we won't get far on criticism only without alternatives to
consider. Don't you think?

[Gavin Lambert]

On 30/05/2016 15:16, Vladimir Batov wrote:
> So, in your view, should we try and have pimpl in Boost? I think we
> should as it'd be useful IMO. I proposed the one I use. I am not married
> to it. If you have a design which makes you happy, let's consider it...
> Otherwise, we won't get far on criticism only without alternatives to
> consider. Don't you think?

As I said at the start, yes, I think this has potential usefulness and
it would be nice to get something like it into Boost.

I'm just more dubious about the specific design as it stands, so I'm
trying to open debate about it, with the goal of finding an improvement,
not shutting anything down. Perhaps I'm just not very good at getting
that across.

As it's the template specialisation which causes most of my concern, my
main query is whether it's really necessary or if it could be structured
differently to avoid this instead.

> I feel that you make very ambitious generalizations about how users
> will feel and how ugly in your view the code will be of those who
> might decide to use the proposed pimpl.

That's certainly possible. But perhaps an illustration might better
explain my perspective (or possibly let others write them off as
personal coding style issues -- that's possible too). Taking the Book
example from the docs (which I know you said are out of date, but it
seems like the broad strokes still apply) and extrapolating it for code
that uses a local namespace and with pimpl in the boost namespace, we
get something like this:

#include ...
#include ...

using std::string;

namespace library
{
namespace detail
{
// some implementation detail definitions, used
// by the pimpl implementation but not part of it
}
}

namespace boost
{
template<> struct pimpl<library::Book>::implementation
{
implementation(string const& the_title,
string const& the_author)
: title(the_title), author(the_author), price(0)
{}
...
bool check_isbn_10_digit() { ... }
...
string title;
string author;
int price;
};
}

namespace library
{
Book::Book(string const& title, string const& author)
: pimpl_type(title, author)
{}

string const&
Book::author() const
{
return (*this)->author;
}

...
}

Namespaces are broken up and repeated, and the implementation of
check_isbn_10_digit() and other such private methods is outside of the
library namespace so it can be a bit awkward to refer to things that you
assume are in scope but aren't actually. It's also awkward if you don't
want to define these methods inline for some reason.

You could skip the namespace at the bottom and explicitly implement the
public methods with library::Book::* instead, but I don't think this is
an improvement; it just makes return type specification more awkward.

[Vladimir Batov]

On 2016-05-30 14:18, Gavin Lambert wrote:
> On 30/05/2016 15:16, Vladimir Batov wrote:
>> So, in your view, should we try and have pimpl in Boost? I think we

>> should as it'd be useful IMO...

>
> As I said at the start, yes, I think this has potential usefulness and
> it would be nice to get something like it into Boost.

Excellent.

> ...

> As it's the template specialisation which causes most of my concern,
> my main query is whether it's really necessary or if it could be
> structured differently to avoid this instead.
>

> ... Taking the Book example from the docs...
> ... we get something like this:
>
> namespace library
> {
> namespace detail
> {
> ...

> }
> }
>
> namespace boost
> {
> template<> struct pimpl<library::Book>::implementation
> {
> implementation(string const& the_title,
> string const& the_author)
> : title(the_title), author(the_author), price(0)
> {}
> ...
> bool check_isbn_10_digit() { ... }
> ...
> string title;
> string author;
> int price;
> };
> }
>
> namespace library
> {
> Book::Book(string const& title, string const& author)
> : pimpl_type(title, author)
> {}
>
> string const&
> Book::author() const
> {
> return (*this)->author;
> }
> ...
> }

> ...

Yes, I understand. That's why I said before and still insist now that
that's style... because I personally do not like

namespace library
{
Book::Book(string const& title, string const& author)
: pimpl_type(title, author)
{}
}

as you move your code to the right and IMO just waste the space (you
fill it with blanks instead of the code). So, I write

template<> struct boost::pimpl<library::Book>::implementation
{
....

}

library::Book::Book(string const& title, string const& author)
: pimpl_type(title, author)
{

...
}

Then, you might say you do not like fully qualified
library::Book::Book(), etc... which in my view is, well, style... for me
the "like" and "do not like" are usually the style triggers.

[Vladimir Batov]

On 2016-05-30 12:36, Emil Dotchevski wrote:
> My 2c:
>
> In foo.h, instead of
>
> class foo
> {
> class pimpl
> pimpl * p_;
> public:
> foo();
> ~foo();
> void do_something();
> };
>
> It's better to do it the C way: simply leave foo incomplete:
>
> struct foo;
> foo * create_foo();
> void destroy_foo( foo * );
> void do_something( foo * );
>
> The above is much improved using shared_ptr:
>
> struct foo;
> shared_ptr<foo> create_foo();
> void do_something( foo * );

If you are saying that pimpl is conceptually simple, then I agree.
std::unique_ptr is conceptually simple as well.

If you are saying that the snippets you provided are sufficient, then I
have to disagree. Say, pimpl in the context of polymorphic hierarchies
or value pimpls come to mind.

[Rob Stewart]

On May 30, 2016 1:42:39 AM EDT, Vladimir Batov <Vladimi...@constrainttec.com> wrote:
>On 2016-05-30 14:18, Gavin Lambert wrote:
>> On 30/05/2016 15:16, Vladimir Batov wrote:
>>> So, in your view, should we try and have pimpl in Boost? I think we
>>> should as it'd be useful IMO...
>>
>> As I said at the start, yes, I think this has potential usefulness
>and
>> it would be nice to get something like it into Boost.
>
>Excellent.

I, too, think that something that simplifies The Pimpl Idiom could be a good addition to Boost.

Adding The Pimpl Idiom to a class means splitting the implements across two class: one public and one private. Even then, the private class is, normally nested in the public class so they are tightly related. With your library, that split is carried further by splitting the code across two distinct namespaces.

That is, indeed, a style issue, since the code still does the same work. Nevertheless it seems wrong to write the implementation details of one's own class in the (proposed) boost namespace.

I've not given any thought to alternatives, so I can only criticize your solution at present.

___
Rob

(Sent from my portable computation engine)

[Vladimir Batov]

On 05/31/2016 12:56 AM, Rob Stewart wrote:
> On May 30, 2016 1:42:39 AM EDT, Vladimir Batov <Vladimi...@constrainttec.com> wrote:
>> On 2016-05-30 14:18, Gavin Lambert wrote:
>>
>>> it would be nice to get something like it into Boost.
>> Excellent.
> I, too, think that something that simplifies The Pimpl Idiom could be a good addition to Boost.

Thank you, Robert, for chiming in. Much appreciated. Maybe this time
we'll be able to get enough momentum and to actually get something
tangible out of it.

> Adding The Pimpl Idiom to a class means splitting the implements
> across two class: one public and one private. Even then, the private
> class is, normally nested in the public class so they are tightly
> related. With your library, that split is carried further by splitting
> the code across two distinct namespaces. That is, indeed, a style
> issue, since the code still does the same work. Nevertheless it seems
> wrong to write the implementation details of one's own class in the
> (proposed) boost namespace.

Yes, I hear you. And, yes, from a certain angle that might seem wrong...
On the other hand, from a certain angle anything might seem wrong... :-)
Is it a serious design flaw? I personally do not think so. More so, I
personally see it from a different angle. Namely, I read the code below
as "Book is declared as a pimpl; then I define the implementation of
that pimpl-Book". Seems sensible and even natural:

struct Book : public pimpl<Book>::shared {};

template<> pimpl<Book>::implementation {};

So, when it is simply read (rather than mechanically dissected) it does
not seem that wrong at all... to me :-)

The "boost::" prefix may or may not be explicitly present... It is up to
coder's preferences and style.

> I've not given any thought to alternatives, so I can only criticize
> your solution at present.

Thank you for mentioning it. I'll hijack your prop to re-iterate that I
very much hope we all can keep the discussion *constructive* and
ultimately moving forward. That IMO means picking the best available
solution... and "no solution" is not one of the choices. I am sure we'll
never get anything that everyone likes. However, a solution is immensely
better than no solution. IMO it's important that we get *something*
in... The seed that the library will grow, mature, improve from. We have
plenty of libs that went through several incompatible changes... It did
not make them worse. It made them better. I am not sure if all those
improvements (for everyone's benefits) were possible if those libs did
not have the visibility, exposure and scrutiny which come with the Boost
tag.

[Gavin Lambert]

On 31/05/2016 09:37, Vladimir Batov wrote:
> Yes, I hear you. And, yes, from a certain angle that might seem wrong...
> On the other hand, from a certain angle anything might seem wrong... :-)
> Is it a serious design flaw? I personally do not think so. More so, I
> personally see it from a different angle. Namely, I read the code below
> as "Book is declared as a pimpl; then I define the implementation of
> that pimpl-Book". Seems sensible and even natural:
>
> struct Book : public pimpl<Book>::shared {};
>
> template<> pimpl<Book>::implementation {};
>
> So, when it is simply read (rather than mechanically dissected) it does
> not seem that wrong at all... to me :-)
>
> The "boost::" prefix may or may not be explicitly present... It is up to
> coder's preferences and style.

The trouble is that the syntax you've written above is not legal C++.
In order to provide the implementation of the impl class (assuming that
pimpl is in the boost namespace), it *must* be done thusly:

// close any other namespace blocks first
namespace boost
{
template<> struct pimpl<other::ns::Book>::implementation
{
...
};
}

In particular it is *not* legal to use a boost:: prefix (or even to
leave it unprefixed in the presence of a "using namespace boost") -- you
have to put it in a namespace block. Otherwise you are declaring a new
type in a different namespace, not specialising the existing template.

This is the part that makes it feel like you're actually implementing
your class (or at least its private members, anyway) in the boost
namespace, and that feels wrong (unless you're a boost library author of
course).

(You're probably going to say that feelings are style decisions again,
which is true -- but it's still important.)

> Thank you for mentioning it. I'll hijack your prop to re-iterate that I
> very much hope we all can keep the discussion *constructive* and
> ultimately moving forward. That IMO means picking the best available
> solution... and "no solution" is not one of the choices. I am sure we'll
> never get anything that everyone likes. However, a solution is immensely
> better than no solution. IMO it's important that we get *something*
> in... The seed that the library will grow, mature, improve from. We have
> plenty of libs that went through several incompatible changes... It did
> not make them worse. It made them better. I am not sure if all those
> improvements (for everyone's benefits) were possible if those libs did
> not have the visibility, exposure and scrutiny which come with the Boost
> tag.

I agree -- but using template specialisation or not is a fundamental
design decision. It's not possible to incrementally improve it later,
only to tear it out and replace it with something different. My point
is that if we can figure out what that something different should be,
then now is the best time to make that change, before it gets users
whose code would break from changes.

Exactly what that change would look like, I'm not sure; I'm hoping that
others will provide feedback and suggestions on this point.

One possibility is that the template specialisation could just be for a
traits type that provides a typedef for the "real" implementation class.
Not sure whether that really improves anything though; it might just
make it more wordy.

[Vladimir Batov]

OK. Clearly in my code it is not in the boost namespace for obvious
reasons... Now, we can over-dramatize things indefinitely but what
ultimately matters to me is if the thing works for me. If you insist
that the world will end if pimpl is not in the boost namespace, then
I'll probably disagree. For those who'll go shirtless unless a shirt has
the Gucci tag or it must be inside the boost namespace I do the
following (just tested with gcc-4.8):

template<class user_type>
struct pimpl // It's outside boost! We all gonna die.
{
}

namespace boost
{
// Hmm, we might probably survive after all.
template<class user_type> using pimpl = ::pimpl<user_type>;
}

// Look, ma! It all is in the boost namespace. It feels so good.
struct Base : public boost::pimpl<Base>::shared { ... };

template<>
struct boost::pimpl<Book>::implementation {...};

>
>> Thank you for mentioning it. I'll hijack your prop to re-iterate that I
>> very much hope we all can keep the discussion *constructive* and
>> ultimately moving forward. That IMO means picking the best available
>> solution... and "no solution" is not one of the choices. I am sure we'll
>> never get anything that everyone likes. However, a solution is immensely
>> better than no solution. IMO it's important that we get *something*
>> in... The seed that the library will grow, mature, improve from. We have
>> plenty of libs that went through several incompatible changes... It did
>> not make them worse. It made them better. I am not sure if all those
>> improvements (for everyone's benefits) were possible if those libs did
>> not have the visibility, exposure and scrutiny which come with the Boost
>> tag.
>
> I agree -- but using template specialisation or not is a fundamental
> design decision.

I personally love those "I agree -- but". If "we agree", we move
forward. If "we agree but", we stay where we were and keep arguing. That
is, it's essentially "I disagree", isn't? :-)

From my experience "template specialisation" is essential to create a
*unique type*.

> It's not possible to incrementally improve it later, only to tear it
> out and replace it with something different.

If it has to be "tear and replace", so be it. People did that for Boost
libs and the Standard. We all adjusted and moved on.

> My point is that if we can figure out what that something different
> should be, then now is the best time to make that change, before it
> gets users whose code would break from changes.

I hear you. Not to be rude but I am still waiting for "that something
different" to consider. Without it it's a one-way endless and unfair
game -- you have all the fun criticizing, I sweat trying to make you happy.

As for the "best time", then I feel you tend to over-dramatize
unnecessarily. If we offer something to wide audience (as part of
Boost), that wider audience will evaluate the design with everything
else following. If that means the current-design acceptance, good. If
that means coming up with something better, even better. Seems like
win-win for everyone. Don't you agree?

[Emil Dotchevski]

Value pimpl, sure, if you need it.

Polymorphism -- the snippets I provided are sufficient, virtual function
calls work just fine. For example, in the CPP file you could define
do_something (see above declaration) like so:

void do_something( foo * p )
{
p->do_something(); //virtual
}

Implicit/static and dynamic casts between polymorphic types work just as
well, since the types left incomplete in the header file are complete in
the cpp file. For example, in the header we could say:

struct foo;
struct bar;
bar * to_bar( foo * );
foo * to_foo( bar * );

And in the cpp:

struct bar
{
....
};

struct foo: bar
{
....
};

bar * to_bar( foo * p )
{
return p;
}

foo * to_foo( bar * p )
{
assert(dynamic_cast<foo *>(p)!=0);
return static_cast<foo *>(p);;
}

Emil

[Vladimir Batov]

On 2016-05-31 13:41, Emil Dotchevski wrote:
> On Sun, May 29, 2016 at 10:47 PM, Vladimir Batov <
> Vladimi...@constrainttec.com> wrote:
>> On 2016-05-30 12:36, Emil Dotchevski wrote:
>>
>>> It's better to do it the C way: simply leave foo incomplete:
>>>
>>> struct foo;
>>> foo * create_foo();
>>> void destroy_foo( foo * );
>>> void do_something( foo * );
>>>
>>> The above is much improved using shared_ptr:
>>>
>>> struct foo;
>>> shared_ptr<foo> create_foo();
>>> void do_something( foo * );
>>

> ...

> Polymorphism -- the snippets I provided are sufficient, virtual
> function
> calls work just fine. For example, in the CPP file you could define
> do_something (see above declaration) like so:
>
> void do_something( foo * p )
> {
> p->do_something(); //virtual
> }

> ...

Indeed. You right... Once we take the methods out to be free-standing
functions, there is nothing left of the interface/proxy class but
shared_ptr<foo>... I have to admit it never occurred to me... :-) Not
sure if I am ready to do things your way :-) but you are most certainly
correct. Thank you.

[Emil Dotchevski]

On Mon, May 30, 2016 at 11:22 PM, Vladimir Batov <

Vladimi...@constrainttec.com> wrote:

> On 2016-05-31 13:41, Emil Dotchevski wrote:
>
>> On Sun, May 29, 2016 at 10:47 PM, Vladimir Batov <

>> ...
>> Polymorphism -- the snippets I provided are sufficient, virtual function
>> calls work just fine. For example, in the CPP file you could define
>> do_something (see above declaration) like so:
>>
>> void do_something( foo * p )
>> {
>> p->do_something(); //virtual
>> }
>> ...
>>
>
> Indeed. You right... Once we take the methods out to be free-standing
> functions, there is nothing left of the interface/proxy class but
> shared_ptr<foo>... I have to admit it never occurred to me... :-) Not sure
> if I am ready to do things your way :-) but you are most certainly correct.
> Thank you.

Wow I must admit this surprises me, the resistance I usually get is much
stronger. :)

The problem from language design point of view is that in C++ the "dot
syntax" can only be used with member functions, but member functions have
access to the private data, which means that they must participate in the
type definitions (encapsulation), which means they can't be called without
the type being complete. The C++ solution to this problem is to use only
abstract types with no data members in header files, but that has virtual
function call overhead compared to the C-style approach.

It would have been useful for C++ to allow the declaration of non-friend
"member" functions outside of the type definition. This would require no
change in syntax.

Thanks,
Emil

[Chris Glover]

>
>
> Wow I must admit this surprises me, the resistance I usually get is much
> stronger. :)
>
>

Then I will offer up one point of resistance.

I sometimes don't pimpl an entire object. This might be because I have
templates in the interface or because I need the functions to inline. Your
solution doesn't allow me to do that.

It also forces a dynamic allocation which we already discussed could be
avoided by using an internal buffer.

Other than those thigns, I like this approach.

-- chris

[Emil Dotchevski]

On Tue, May 31, 2016 at 12:09 PM, Chris Glover <c.d.g...@gmail.com> wrote:

> I sometimes don't pimpl an entire object. This might be because I have
> templates in the interface or because I need the functions to inline. Your
> solution doesn't allow me to do that.
>

I agree that being able to un-pimple a subset of the members in the face of
profiler evidence that inlining is necessary is important. I'm not sure why
do you say that "my approach" doesn't support that. One possibility is to
use inheritance in the cpp file:

header:

struct foo { int critical_; };
shared_ptr<foo> create_foo();
inline void use_foo_critical( foo * p ) { ++p->critical_; }
void use_foo_not_so_critical( foo * );

cpp:

namespace
{
struct foo_: foo
{
int not_so_critiral_;
};
}

shared_ptr<foo> create_foo()
{
return shared_ptr<foo>(new foo_);
}

void use_foo_not_so_critical( foo * p )
{
foo_ * q=static_cast<foo_ *>(p);
++q->not_so_critical_;

}


> It also forces a dynamic allocation which we already discussed could be
> avoided by using an internal buffer.
>

This is actually the main motivation for using shared_ptr directly in the
header -- to allow for custom allocators in the create_foo function, which
can completely eliminate dynamic allocations. Generally, if you're
concerned about dynamic allocations yet you're not using shared_ptr, you're
doing it wrong. :)

If we didn't care about this point of control, then it'd be more portable
to remove shared_ptr from the header and simply define a C interface:

struct foo;
foo * create_foo();
void destroy_foo( foo * );

(In this case a C++ user can still use shared_ptr with a custom deleter, at
the cost of an extra allocation.)

OTOH do note that in the header we may leave shared_ptr incomplete and
still declare a shared_ptr factory function. This leads to extremely lean
header files:

namespace boost { template <class> class shared_ptr; }
struct foo;
boost::shared_ptr<foo> create_foo();
void use_foo( foo * );

Emil

[charleyb123 .]

>
> The problem from language design point of view is that in C++ the "dot
> syntax" can only be used with member functions, but member functions have
> access to the private data, which means that they must participate in the
> type definitions (encapsulation), which means they can't be called without
> the type being complete. The C++ solution to this problem is to use only
> abstract types with no data members in header files, but that has virtual
> function call overhead compared to the C-style approach.

This is why I'm such a fan of CRTP (curiously-recurring-template-pattern),
where it should work for this PIMPL design with no virtual and no overhead.

I've been quiet on this thread, but:

*- I like the PIMPL pattern very much
*- I like your approach
*- I want no overhead
*- I vote CRTP
*- CRTP also allows template-member-functions (requested by Chris, also
desired by me)
*- I'm generally disappointed with namespaces

--charley


On Tue, May 31, 2016 at 1:03 PM, Emil Dotchevski <emildot...@gmail.com>
wrote:

[Chris Glover]

>
>
> I agree that being able to un-pimple a subset of the members in the face of
> profiler evidence that inlining is necessary is important. I'm not sure why
> do you say that "my approach" doesn't support that. One possibility is to
> use inheritance in the cpp file:
>
> header:
>
> struct foo { int critical_; };
> shared_ptr<foo> create_foo();
> inline void use_foo_critical( foo * p ) { ++p->critical_; }
> void use_foo_not_so_critical( foo * );
>
>

Yes, it seems I missed this. Thanks!

> OTOH do note that in the header we may leave shared_ptr incomplete and
> still declare a shared_ptr factory function. This leads to extremely lean
> header files:
>
> namespace boost { template <class> class shared_ptr; }
> struct foo;
> boost::shared_ptr<foo> create_foo();
> void use_foo( foo * );
>
> Emil
>

You're making a very compelling argument. Actually strong enough that it
makes me want the uniform call syntax when I was previously on the fence
about that feature.

-- chris

[Emil Dotchevski]

On Tue, May 31, 2016 at 2:29 PM, Chris Glover <c.d.g...@gmail.com> wrote:

> > OTOH do note that in the header we may leave shared_ptr incomplete and
> > still declare a shared_ptr factory function. This leads to extremely lean
> > header files:
> >
> > namespace boost { template <class> class shared_ptr; }
> > struct foo;
> > boost::shared_ptr<foo> create_foo();
> > void use_foo( foo * );
>

> You're making a very compelling argument. Actually strong enough that it
> makes me want the uniform call syntax when I was previously on the fence
> about that feature.
>

I'm generally not in favor of adding stuff to C++. What's the upside in
this case? To be able to say p.do_something() instead of do_something(p),
because the latter offends Java programmers? :)

Emil

[Rob Stewart]

On May 31, 2016 5:10:35 PM EDT, Emil Dotchevski <emildot...@gmail.com> wrote:
>
>I'm not sure why
>do you say that "my approach" doesn't support that. One possibility is
>to use inheritance in the cpp file:
>
>header:
>
>struct foo { int critical_; };
>shared_ptr<foo> create_foo();
>inline void use_foo_critical( foo * p ) { ++p->critical_; }
>void use_foo_not_so_critical( foo * )

Were you leaving encapsulation as an exercise for the reader?

class foo
{
public:
use_critical () { ++critical_; }
protected:
int critical_;
};

Your foo_ can still access critical_.

___
Rob

(Sent from my portable computation engine)

[Rob Stewart]

On May 31, 2016 5:41:54 PM EDT, Emil Dotchevski <emildot...@gmail.com> wrote:
>On Tue, May 31, 2016 at 2:29 PM, Chris Glover <c.d.g...@gmail.com>
>wrote:
>

>> You're making a very compelling argument. Actually strong enough that
>> it makes me want the uniform call syntax when I was previously on the
>> fence about that feature.
>
>I'm generally not in favor of adding stuff to C++. What's the upside in
>this case? To be able to say p.do_something() instead of
>do_something(p),
>because the latter offends Java programmers? :)

The upside is not writing some calls one way and others the other way on the same object, and having to remember which is which.

___
Rob

(Sent from my portable computation engine)

[Josh Juran]

On May 31, 2016, at 5:10 PM, Emil Dotchevski <emildot...@gmail.com> wrote:

> On Tue, May 31, 2016 at 12:09 PM, Chris Glover <c.d.g...@gmail.com> wrote:
>
>> I sometimes don't pimpl an entire object. This might be because I have
>> templates in the interface or because I need the functions to inline. Your
>> solution doesn't allow me to do that.
>>
>

> One possibility is to use inheritance in the cpp file:
>
> header:
>
> struct foo { int critical_; };
>

> cpp:
>
> namespace
> {
> struct foo_: foo
> {
> int not_so_critiral_;
> };
> }

Since this uses a derived class instead of a pointer, I've been calling this the "dimpl" idiom. :-)

Josh

[Emil Dotchevski]

On Tue, May 31, 2016 at 2:45 PM, Rob Stewart <rste...@ptd.net> wrote:

> On May 31, 2016 5:10:35 PM EDT, Emil Dotchevski <emildot...@gmail.com>
> wrote:
> >
> >I'm not sure why
> >do you say that "my approach" doesn't support that. One possibility is
> >to use inheritance in the cpp file:
> >
> >header:
> >
> >struct foo { int critical_; };
> >shared_ptr<foo> create_foo();
> >inline void use_foo_critical( foo * p ) { ++p->critical_; }
> >void use_foo_not_so_critical( foo * )
>
> Were you leaving encapsulation as an exercise for the reader?
>
> class foo
> {
> public:
> use_critical () { ++critical_; }
> protected:
> int critical_;
> };
>
> Your foo_ can still access critical_.
>

Right, though my preference is as follows.

//Public interface:

namespace boost { template <class> class shared_ptr; }

class foo;
boost::shared_ptr<foo> create_foo();
void use_foo_critical( foo * );
void use_foo_not_so_critical( foo * );

//Implementation details:

class foo
{
foo( foo const & );
foo & operator=( foo const & );
int critical_;
friend void use_foo_critical( foo * );
protected:
foo();
~foo();

};
inline void use_foo_critical( foo * p ) { ++p->critical_; }

Emil

[Emil Dotchevski]

On Tue, May 31, 2016 at 2:48 PM, Rob Stewart <rste...@ptd.net> wrote:

> On May 31, 2016 5:41:54 PM EDT, Emil Dotchevski <emildot...@gmail.com>
> wrote:
> >On Tue, May 31, 2016 at 2:29 PM, Chris Glover <c.d.g...@gmail.com>
> >wrote:
> >
> >> You're making a very compelling argument. Actually strong enough that
> >> it makes me want the uniform call syntax when I was previously on the
> >> fence about that feature.
> >
> >I'm generally not in favor of adding stuff to C++. What's the upside in
> >this case? To be able to say p.do_something() instead of
> >do_something(p),
> >because the latter offends Java programmers? :)
>
> The upside is not writing some calls one way and others the other way on
> the same object, and having to remember which is which.
>

So, don't use the dot syntax. :)

Emil

[Vladimir Batov]

On 2016-06-01 07:58, Emil Dotchevski wrote:
> On Tue, May 31, 2016 at 2:48 PM, Rob Stewart <rste...@ptd.net> wrote:
>> The upside is not writing some calls one way and others the other way
>> on
>> the same object, and having to remember which is which.
>
> So, don't use the dot syntax. :)

Indeed. If fact, with "your" approach we can't "use the dot syntax"...
And that's the "problem". :-) It's not a criticism. It's merely a fair
observation. I myself 've come to C++ with a considerable C experience
and I do not "cringe" seeing "your" free-function-based API. Others
might object stronger as we are entering the "style area" where people
fight tooth and nail over nothing. :-) Even I must say that from the
"purist" point of view forcing free-function API kinda violates the very
basic OO paradigm -- the association between data and behavior. From
practical point of view it's "meh, big deal".

Please do not get me wrong. I am not criticizing your approach. In fact,
to me it feels surprisingly potent just using the tools we've had
"forever". I feel that it is unlikely to "fly" with "general programming
population". :-)

[Gavin Lambert]

On 1/06/2016 09:12, charleyb123 . wrote:
> This is why I'm such a fan of CRTP (curiously-recurring-template-pattern),
> where it should work for this PIMPL design with no virtual and no overhead.
>
> I've been quiet on this thread, but:
>
> *- I like the PIMPL pattern very much
> *- I like your approach
> *- I want no overhead
> *- I vote CRTP
> *- CRTP also allows template-member-functions (requested by Chris, also
> desired by me)
> *- I'm generally disappointed with namespaces

This already uses CRTP, eg:

struct T : public pimpl<T>::shared {...};

Is there some other application of this that you actually meant instead?

[rstewart]

"Emil Dotchevski" <emildot...@gmail.com> wrote:
> On Tue, May 31, 2016 at 2:48 PM, Rob Stewart <rste...@ptd.net> wrote:
> > On May 31, 2016 5:41:54 PM EDT, Emil Dotchevski <emildot...@gmail.com>
> > wrote:
> > >On Tue, May 31, 2016 at 2:29 PM, Chris Glover <c.d.g...@gmail.com>
> > >wrote:
> > >
> > >> You're making a very compelling argument. Actually strong enough that
> > >> it makes me want the uniform call syntax when I was previously on the
> > >> fence about that feature.
> > >
> > >I'm generally not in favor of adding stuff to C++. What's the upside in
> > >this case? To be able to say p.do_something() instead of
> > >do_something(p), because the latter offends Java programmers? :)
> >
> > The upside is not writing some calls one way and others the other way on
> > the same object, and having to remember which is which.
> >
> So, don't use the dot syntax. :)

Not all functions can be non-members.

[Emil Dotchevski]

On Tue, May 31, 2016 at 3:15 PM, Vladimir Batov <
Vladimi...@constrainttec.com> wrote:

> On 2016-06-01 07:58, Emil Dotchevski wrote:
>
>> On Tue, May 31, 2016 at 2:48 PM, Rob Stewart <rste...@ptd.net> wrote:
>>
>>> The upside is not writing some calls one way and others the other way on
>>> the same object, and having to remember which is which.
>>>
>>
>> So, don't use the dot syntax. :)
>>
>

> Even I must say that from the "purist" point of view forcing free-function
> API kinda violates the very basic OO paradigm -- the association between
> data and behavior. From practical point of view it's "meh, big deal".
>

C++ provides alternative mechanisms to express that association, for
example:

namespace boost { template <class> class shared_ptr; };

namespace file
{
struct handle;
boost::shared_ptr<handle> open( char const * name );
void read( handle *, void * buf, size_t size );
}

As a bonus, ADL allows unqualified calls to read (but, obviously, not to
open.)

That said, the most important feature of object-oriented design is data
encapsulation, which is actually stronger if types are left incomplete in
interfaces, compared to using private/protected.

> Please do not get me wrong. I am not criticizing your approach. In fact,
> to me it feels surprisingly potent just using the tools we've had
> "forever". I feel that it is unlikely to "fly" with "general programming
> population". :-)

Yes, I'm aware of that. It doesn't mean they have a point though. :)

I just can't support including yet another can of worms in the already
complicated name lookup/implicit instantiation/overload resolution process.
A compromise would be to allow the definition of non-friend members, like
this:

struct foo;
void foo::do_something();

This language change seems a lot safer than messing with the overload
resolution. Perhaps I'm wrong. Either way, I don't consider "but I want to
use the dot syntax" a very compelling argument.

Emil

[rstewart]

That doesn't provide inline access to those with access only to the header.

[Emil Dotchevski]

On Tue, May 31, 2016 at 4:05 PM, rstewart <rste...@ptd.net> wrote:

> "Emil Dotchevski" <emildot...@gmail.com> wrote:
> > On Tue, May 31, 2016 at 2:48 PM, Rob Stewart <rste...@ptd.net> wrote:
> > > On May 31, 2016 5:41:54 PM EDT, Emil Dotchevski <
> emildot...@gmail.com>
> > > wrote:
> > > >On Tue, May 31, 2016 at 2:29 PM, Chris Glover <c.d.g...@gmail.com>
> > > >wrote:
> > > >
> > > >> You're making a very compelling argument. Actually strong enough
> that
> > > >> it makes me want the uniform call syntax when I was previously on
> the
> > > >> fence about that feature.
> > > >
> > > >I'm generally not in favor of adding stuff to C++. What's the upside
> in
> > > >this case? To be able to say p.do_something() instead of
> > > >do_something(p), because the latter offends Java programmers? :)
> > >
> > > The upside is not writing some calls one way and others the other way
> on
> > > the same object, and having to remember which is which.
> > >
> > So, don't use the dot syntax. :)
>
> Not all functions can be non-members.
>

Do you mean e.g. virtual functions? They can be hidden behind free function
wrappers.

Emil

[Emil Dotchevski]

I meant that whole thing being in the header, which is required for inline
access (within the CPP file inline is next to pointless since the compiler
can easily inline any function.) The point I'm making with
"//implementation details" is that the fact that use_foo_critical is
defined inline, as well as the exact definition of foo itself, is not part
of the interface and thus subject to change.

Emil

[Vladimir Batov]

On 2016-06-01 07:10, Emil Dotchevski wrote:
> ...

> header:
>
> struct foo { int critical_; };
> shared_ptr<foo> create_foo();
> inline void use_foo_critical( foo * p ) { ++p->critical_; }
> void use_foo_not_so_critical( foo * );
>
> cpp:
>
> namespace
> {
> struct foo_: foo
> {
> int not_so_critiral_;
> };
> }
>
> shared_ptr<foo> create_foo()
> {
> return shared_ptr<foo>(new foo_);
> }
>
> void use_foo_not_so_critical( foo * p )
> {
> foo_ * q=static_cast<foo_ *>(p);
> ++q->not_so_critical_;
> }

> ...

Hmm, we probably need to adjust your example (admittedly written in a
hurry) -- "foo" needs to be virtual (given we wanted efficiency),
static_cast kills all C++ advances in type-safety... So, probably it
should better be

struct foo_;
struct foo { int critical_; shared_ptr<foo_>; };
shared_ptr<foo> create_foo();

Then, when you throw in access restrictions (public/private) as Robert
mentioned, those free functions need to be declared friends -- a fix due
to initial "design simplification" -- violation of the association
between data and behavior.

... And that's where the free-function-based API approach unfortunately
seems to begin to start faltering as the beauty of simplicity is
gradually fading away.

Please do not take it as a criticism. I am just thinking aloud to see
how far your approach can take us (how "scalable" it is :-)) and I hope
you'll correct me if I veer in the wrong direction.

[Vladimir Batov]

On 2016-06-01 05:03, Emil Dotchevski wrote:
> ...

> The problem from language design point of view is that in C++ the "dot
> syntax" can only be used with member functions, but member functions
> have
> access to the private data, which means that they must participate in
> the
> type definitions (encapsulation), which means they can't be called
> without
> the type being complete. The C++ solution to this problem is to use
> only
> abstract types with no data members in header files, but that has
> virtual
> function call overhead compared to the C-style approach.
>
> It would have been useful for C++ to allow the declaration of
> non-friend
> "member" functions outside of the type definition. This would require
> no
> change in syntax.

Interesting... and liberating... and dangerous... :-) Aren't you
suggesting to actually officially support encapsulation violation? Say,
by your book I can write a new free-standing "member" function to your
class and cause quite a bit of mischief in there.

[Emil Dotchevski]

On Tue, May 31, 2016 at 4:21 PM, Vladimir Batov <
Vladimi...@constrainttec.com> wrote:

> On 2016-06-01 07:10, Emil Dotchevski wrote:
>
>> ...
>> header:
>>
>> struct foo { int critical_; };
>> shared_ptr<foo> create_foo();
>> inline void use_foo_critical( foo * p ) { ++p->critical_; }
>> void use_foo_not_so_critical( foo * );
>>
>> cpp:
>>
>> namespace
>> {
>> struct foo_: foo
>> {
>> int not_so_critiral_;
>> };
>> }
>>
>> shared_ptr<foo> create_foo()
>> {
>> return shared_ptr<foo>(new foo_);
>> }
>>
>> void use_foo_not_so_critical( foo * p )
>> {
>> foo_ * q=static_cast<foo_ *>(p);
>> ++q->not_so_critical_;
>> }
>> ...
>>
>
> Hmm, we probably need to adjust your example (admittedly written in a
> hurry) -- "foo" needs to be virtual (given we wanted efficiency),

You mean polymorphic? It could be.

> static_cast kills all C++ advances in type-safety...

The use of static_cast in this case is perfectly safe, since there is no
way to get to undefined behavior using the public interface only. When I
say "there is no way" I mean that static type checking isn't butchered at
all.

> So, probably it should better be
>
> struct foo_;
> struct foo { int critical_; shared_ptr<foo_>; };
> shared_ptr<foo> create_foo();
>

I disagree, I don't see the benefit of exposing foo_ in the header.

Emil

[Gavin Lambert]

On 1/06/2016 11:05, rstewart wrote:
>>> The upside is not writing some calls one way and others the other way on
>>> the same object, and having to remember which is which.
>>>
>> So, don't use the dot syntax. :)
>
> Not all functions can be non-members.

Operators can be; so the only types of functions I'm aware of that must
be members are the constructors (regular, copy, and move).

But constructors simply don't exist with this design, since you use
factory methods in place of regular constructors, and copy/move
construction is not possible with an incomplete type (which is all that
external code will ever see).

Moving is not really an issue -- you can move the shared_ptr instead.
The same applies to shallow copying (copying the pointer rather than the
underlying object).

If you want to be able to deep copy the object then this requires an
explicit API (typically a "shared_ptr<T> clone(T*)" factory method).
This is the reverse of traditional C++ objects which are copyable by
default -- these are copyable only where explicitly permitted. I'm sure
there will be people who argue that this is a good thing and those that
argue that it isn't. :)


I've used a similar coding style when writing object-oriented C code,
although I admit it would feel weirder doing it in C++ (although this
has some natural benefits, such as overloading).

[Emil Dotchevski]

On Tue, May 31, 2016 at 4:42 PM, Vladimir Batov <
Vladimi...@constrainttec.com> wrote:

> On 2016-06-01 05:03, Emil Dotchevski wrote:
>
>> ...
>> The problem from language design point of view is that in C++ the "dot
>> syntax" can only be used with member functions, but member functions have
>> access to the private data, which means that they must participate in the
>> type definitions (encapsulation), which means they can't be called without
>> the type being complete. The C++ solution to this problem is to use only
>> abstract types with no data members in header files, but that has virtual
>> function call overhead compared to the C-style approach.
>>
>> It would have been useful for C++ to allow the declaration of non-friend
>> "member" functions outside of the type definition. This would require no
>> change in syntax.
>>
>
> Interesting... and liberating... and dangerous... :-) Aren't you
> suggesting to actually officially support encapsulation violation?

No, I said non-friend. Look, there is no semantic difference between
allowing non-friend member functions to be added without changes to the
type definition on one hand, and allowing non-friend free functions to be
added without changes to the type definition on the other hand. Neither
breaks the encapsulation; the difference is entirely syntactic.

> Say, by your book I can write a new free-standing "member" function to
> your class and cause quite a bit of mischief in there.

Not without having access to the private/protected stuff, which you
wouldn't have.

Emil

[Emil Dotchevski]

On Tue, May 31, 2016 at 4:47 PM, Gavin Lambert <gav...@compacsort.com>
wrote:

> On 1/06/2016 11:05, rstewart wrote:
>
>> The upside is not writing some calls one way and others the other way on
>>>> the same object, and having to remember which is which.
>>>>
>>>> So, don't use the dot syntax. :)
>>>
>>
>> Not all functions can be non-members.
>>
>
> Operators can be; so the only types of functions I'm aware of that must be
> members are the constructors (regular, copy, and move).
>
> But constructors simply don't exist with this design

Not in the public interface, but they would exist in the private
implementation. Encapsulation is great, all that goodness with constructors
establishing invariants, member functions maintaining invariants,
exceptions enforcing postconditions -- it's all good, but it is now an
implementation detail; all that code lives in the CPP file.

> Moving is not really an issue -- you can move the shared_ptr instead. The
> same applies to shallow copying (copying the pointer rather than the
> underlying object).
>

Yes, but that's different. It's more precise to say that one can't move or
copy an incomplete type.

Emil

[Gavin Lambert]

On 1/06/2016 11:48, Emil Dotchevski wrote:
> No, I said non-friend. Look, there is no semantic difference between
> allowing non-friend member functions to be added without changes to the
> type definition on one hand, and allowing non-friend free functions to be
> added without changes to the type definition on the other hand. Neither
> breaks the encapsulation; the difference is entirely syntactic.

This is basically what .NET extension methods do -- they're static
functions outside of a class declared with some initial parameter of a
particular type. They can be invoked either as static methods passing
the parameters exactly as declared, or as if they were member functions
where the initial parameter is removed from the argument list and placed
before the dot instead.

eg. public static int Sum(this IEnumerable<int> list), which can be
called just as list.Sum() on any IEnumerable<int>.

It's purely syntactic but allows for some highly readable code.
(Although it's easy to get carried away sometimes.)

[Peter Dimov]

Vladimir Batov wrote:
> On 2016-06-01 05:03, Emil Dotchevski wrote:
>
>> It would have been useful for C++ to allow the declaration of non-friend
>> "member" functions outside of the type definition. This would require no
>> change in syntax.
>
> Interesting... and liberating... and dangerous... :-) Aren't you
> suggesting to actually officially support encapsulation violation?

These are called "extension methods" and they don't violate encapsulation
because they aren't "friends".

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0079r0.pdf

[Gavin Lambert]

On 1/06/2016 11:55, Emil Dotchevski wrote:
> Not in the public interface, but they would exist in the private
> implementation. Encapsulation is great, all that goodness with constructors
> establishing invariants, member functions maintaining invariants,
> exceptions enforcing postconditions -- it's all good, but it is now an
> implementation detail; all that code lives in the CPP file.

Of course.

>> Moving is not really an issue -- you can move the shared_ptr instead. The
>> same applies to shallow copying (copying the pointer rather than the
>> underlying object).
>
> Yes, but that's different. It's more precise to say that one can't move or
> copy an incomplete type.

I did say that.

[Vladimir Batov]

On 2016-06-01 09:07, Emil Dotchevski wrote:
> ...

> That said, the most important feature of object-oriented design is data

> encapsulation, ...

I think it needs corrected. I am pretty sure it is "data and behavior
encapsulation"... and the "behavior" is something that you take out to
be free-standing functions and, actually, violate OO... just saying...
not that I lose sleep over it. :-)

>> ... I feel that it is unlikely to "fly" with "general programming

>> population". :-)
>
> Yes, I'm aware of that. It doesn't mean they have a point though. :)
>
> I just can't support including yet another can of worms in the already
> complicated name lookup/implicit instantiation/overload resolution
> process.

Look who is an orthodox zealot now!.. :-) So, even with understanding
that "general public" is unlikely to do it your way you are not to give
them anything else... Just saying... Not judging... :-)

> A compromise would be to allow the definition of non-friend members,
> like
> this:
>
> struct foo;
> void foo::do_something();

Indeed, that'd be handy to support your pimpl style. I just suspect it
violates encapsulation and, well, it's just not in the language.

> This language change seems a lot safer than messing with the overload
> resolution. Perhaps I'm wrong. Either way, I don't consider "but I want
> to
> use the dot syntax" a very compelling argument.

I do not feel it is as dumb as "but I want to use the dot syntax". I am
beginning to feel that

1) you break the "data+behavior" encapsulation (by taking the methods
out) and we reap the immediate benefits of reducing pimpl to mere
shared_ptr; exciting;

2) now if I want to bring that "data+behavior" association back, I'll
use notional (rather than actual) association wrapping it all in a
namespace; why not;

3) after reading the thread I am under impression that as the
deployments are getting more complex I need to introduce accessibility
(private, public) qualifiers... and the impact of #1 is becoming more
apparent as (due to language limitations you might say) we have to
re-introduce the broken "data+behavior" association via "friends". That
is we seem to be drifting back where we started... but in an
unconventional way... which I am not sure will be eagerly embraced. :-)

[Howard Hinnant]

On May 28, 2016, at 10:35 PM, Vladimir Batov <Vladimi...@constrainttec.com> wrote:
>
> and IMO the proposed design does have advantage over unique_ptr-based pimpl as IMO unique_ptr hardly has any advantage over the raw pointer -- even the destructor has to be explicit non-default and non-inlined.

<nitpick> You keep saying this and it is like fingernails on a chalkboard to me. The correct statement is not that far off of what you’re saying, and does not invalidate your point.

The unique_ptr-based pimpl has to have an outlined destructor, but it can (and should) be defaulted:

Book::~Book() = default;

Howard

[Vladimir Batov]

On 2016-06-01 10:03, Peter Dimov wrote:
> Vladimir Batov wrote:
>> On 2016-06-01 05:03, Emil Dotchevski wrote:
>>
>>> It would have been useful for C++ to allow the declaration of
>>> non-friend "member" functions outside of the type definition. This
>>> would require no change in syntax.
>>
>> Interesting... and liberating... and dangerous... :-) Aren't you
>> suggesting to actually officially support encapsulation violation?
>
> These are called "extension methods" and they don't violate
> encapsulation because they aren't "friends".
>
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0079r0.pdf

Peter, thank you for the link. Interesting... Sometimes (hmm, make it
"always") feel like an old mammoth failing to keep up with the herd...
and grumbling "I am happy with the way it is" :-)

I am not sure if "to allow free function invocation syntax to invoke
member functions and vice-versa" is exactly what Emil wanted. Quoting
from the top --

the "declaration of non-friend "member" functions outside of the type

definition". I read it as Emil wants it to be a "member" but declared
"outside".

[Vladimir Batov]

Howard, my most humble apologies. English is not my first and its
subtleties stubbornly escape me. The least of all I sought to degrade
std::unique_ptr. Truly. Unreservedly. I cringed myself when I read what
I wrote because I left out an important part -- "in pimpl-related
context". And I keep saying those things not because I am trying to
degrade unique_ptr but because IMO unique_ptr was developed for a very
different purpose. It fits the purpose perfectly... it's just not a good
fit for the pimpl idiom... something Sutter suggests in his GotWs over
and over again.

As for

Book::~Book() = default;

I am afraid I have to disagree. I suspect that won't work. Because the
compiler will try to inline ~Book() (i.e. call
unique_ptr<Book::implementation> destructor) and for that
Book::implementation needs to be complete. So, ~Book() even with an
empty body needs to be explicit in the implementation file. Am I right?

[Vladimir Batov]

That's where I think communication breaks. "Friend" to me is essentially
a "member" (just declared outside) as it has access to "private".

So, I interpret "non-friend member" as "non-member member" which my
"computer does not compute" :-)

[Howard Hinnant]

On May 31, 2016, at 8:50 PM, Vladimir Batov <Vladimi...@constrainttec.com> wrote:
>
> On 2016-06-01 10:25, Howard Hinnant wrote:
>> On May 28, 2016, at 10:35 PM, Vladimir Batov
>> <Vladimi...@constrainttec.com> wrote:
>>> and IMO the proposed design does have advantage over unique_ptr-based pimpl as IMO unique_ptr hardly has any advantage over the raw pointer -- even the destructor has to be explicit non-default and non-inlined.
>> <nitpick> You keep saying this and it is like fingernails on a
>> chalkboard to me. The correct statement is not that far off of what
>> you’re saying, and does not invalidate your point.
>> The unique_ptr-based pimpl has to have an outlined destructor, but it
>> can (and should) be defaulted:
>> Book::~Book() = default;
>
> Howard, my most humble apologies. English is not my first and its subtleties stubbornly escape me. The least of all I sought to degrade std::unique_ptr. Truly. Unreservedly. I cringed myself when I read what I wrote because I left out an important part -- "in pimpl-related context". And I keep saying those things not because I am trying to degrade unique_ptr but because IMO unique_ptr was developed for a very different purpose. It fits the purpose perfectly... it's just not a good fit for the pimpl idiom... something Sutter suggests in his GotWs over and over again.

No apologies necessary. I don’t think anyone interprets this as degrading unique_ptr. Everything you say is correct, except for this one little detail: :-)

>
> As for
>
> Book::~Book() = default;
>
> I am afraid I have to disagree. I suspect that won't work. Because the compiler will try to inline ~Book() (i.e. call unique_ptr<Book::implementation> destructor) and for that Book::implementation needs to be complete. So, ~Book() even with an empty body needs to be explicit in the implementation file. Am I right?

Perhaps I wasn’t clear:

It has to be outlined, in the source.cpp:

#include <memory>

class Book
{
struct implementation;
std::unique_ptr<implementation> impl_;
public:
~Book();
Book();
};

int
main()
{
Book b;
}

struct Book::implementation
{
};

Book::~Book() = default;
Book::Book() = default;

It is ok to *define* a special member with “= default” in the source. It is only this tiny misunderstanding I would like to clear up. I agree with you that because of the need to do this outlining there is room for a pimpl toolbox here.

Howard

[Emil Dotchevski]

On Tue, May 31, 2016 at 6:02 PM, Vladimir Batov <

What I mean by non-friend member is something that is currently not
supported in C++, which is defined/declared/called using the member
function syntax, but has no access to the private or protected members of
the class. So:

struct foo; //incomplete
void foo::do_something();

would be semantically the same as:

struct foo; //incomplete
void do_something( foo * );

the only difference being in syntax: the former would be callable using the
dot syntax.

Emil

[charleyb123 .]

On Tue, May 31, 2016 at 4:59 PM, Gavin Lambert <gav...@compacsort.com>
wrote:

> On 1/06/2016 09:12, charleyb123 . wrote:
>
>> This is why I'm such a fan of CRTP (curiously-recurring-template-pattern),
>> where it should work for this PIMPL design with no virtual and no
>> overhead.
>>
>> I've been quiet on this thread, but:
>>
>> *- I like the PIMPL pattern very much
>> *- I like your approach
>> *- I want no overhead
>> *- I vote CRTP
>> *- CRTP also allows template-member-functions (requested by Chris, also
>> desired by me)
>> *- I'm generally disappointed with namespaces
>>
>
> This already uses CRTP, eg:
>
> struct T : public pimpl<T>::shared {...};
>
> Is there some other application of this that you actually meant instead?
>

//"IMPL" interface consistent with "std_impl<>"
template<typename IMPL>
struct std_pimpl
{
using P = typename IMPL::TYPE;
IMPL* impl_;
...
std_pimpl<IMPL>& clone(void) const {
assert(impl_);
return *new std_pimpl(IMPL::Clone(*impl_));
}
...standard pimpl interface forwarding to IMPL::...
};

template<typename D, typename F>
struct std_impl
{
using TYPE = F;
F* impl_;

const F& getImpl(void) const { assert(impl_); return *impl_; }

D& clone(void) const {
return D::Clone(static_cast<const D&>(*this));
}

static D& Clone(const D& clone_from) {
//..default implementation
return *new D(clone_from);
}
...standard impl interface...
};

class Foo {};

struct FooImpl : public std_impl<FooImpl,Foo>
{
using BASE = std_impl<FooImpl,Foo>;
//...
static F& Clone(const Foo& clone_from) {
// Override BASE::Clone()
return *new D(clone_from.getImpl().cloneNew());
}
};

using FooPimp = std_pimpl<FooImpl>;

------------

In short, what I like:

*- some kind of "std_pimpl<>" provides a standardized interface, and
"plumbing"/implementation.
*- CRTP in the "std_impl<>" allows for a zero-cost compile-time
implementation-override.
*- If necessary, implementation can permit "std_pimpl<>" to be
parameterized with merely a "Foo" declaration.

--charley

[Gavin Lambert]

On 1/06/2016 12:35, Vladimir Batov wrote:
> I am not sure if "to allow free function invocation syntax to invoke
> member functions and vice-versa" is exactly what Emil wanted. Quoting
> from the top --
> the "declaration of non-friend "member" functions outside of the type
> definition". I read it as Emil wants it to be a "member" but declared
> "outside".

That's what extension methods are though, as I mentioned in my other post.

They're free functions declared outside of the class (and thus don't
have access to private/protected members) that take the class as their
initial (or possibly only) parameter.

What makes them actual extension methods is that the language allows
them to be called as if they were members -- ie. at the compiler level,
it finds this:

a.f(b);

and if it fails to find a "real" member f on a, but it can find a
compatible free function, then it will pretend that the code had said
this instead:

f(a, b);

This allows external code to define additional methods that aren't
actually part of a class but can be called as if they were, which
enables some quite nice things.

For example, imagine a library that could add string algorithms directly
to std::string (as far as it appears from the call syntax).

Though the real power typically comes from defining extensions for
concepts rather than concrete types, though both are useful.

(An odd side effect of this, which is both good and bad, is that it also
allows well-formed "member" calls on nullptr with a well-defined result.)

[Vladimir Batov]

On 2016-06-01 11:08, Howard Hinnant wrote:
> ...

> It has to be outlined, in the source.cpp:

> ...

> Book::~Book() = default;
> Book::Book() = default;

Uh, yes, indeed. I got it now. I'll adjust the wording in the docs.

Somewhat off the topic.

From efficiency point of view the above is identical to

Book::~Book() {}

or compiler writers do more magic with "=default"? I like "=default" as
I feel it's cleaner language-wise. What weighty reason can I give to a
beginner to prefer "=default" over {}... apart from "I am older,
musclier and your boss" :-)

[Peter Dimov]

Howard Hinnant wrote:

> It has to be outlined, in the source.cpp:
>
> #include <memory>
>
> class Book
> {
> struct implementation;
> std::unique_ptr<implementation> impl_;
> public:
> ~Book();
> Book();
> };

You could probably do

// hpp

class Book
{
struct implementation;
std::unique_ptr<implementation, void(*)(implementation*)> impl_;

public:

Book();
};

// cpp

Book::Book(): impl_( new implementation, checked_delete<implementation> ) {}

[Howard Hinnant]

On May 31, 2016, at 9:51 PM, Vladimir Batov <Vladimi...@constrainttec.com> wrote:
>
> On 2016-06-01 11:08, Howard Hinnant wrote:
>> ...
>> It has to be outlined, in the source.cpp:
>> ...
>> Book::~Book() = default;
>> Book::Book() = default;
>
> Uh, yes, indeed. I got it now. I'll adjust the wording in the docs.

Excellent, thanks.

>
> Somewhat off the topic.
>
> From efficiency point of view the above is identical to
>
> Book::~Book() {}
>
> or compiler writers do more magic with "=default"? I like "=default" as I feel it's cleaner language-wise. What weighty reason can I give to a beginner to prefer "=default" over {}... apart from "I am older, musclier and your boss" :-)

That’s a good question. It doesn’t turn the destructor from non-trivial to trivial. It does not change the noexcept or constexpr status. The only thing it really does is say: I want default semantics here. Maintenance programmers should be more leery of adding superfluous stuff to your destructor.

Howard

[Vladimir Batov]

On 2016-06-01 11:09, Emil Dotchevski wrote:
> ...

> What I mean by non-friend member is something that is currently not
> supported in C++, which is defined/declared/called using the member
> function syntax, but has no access to the private or protected members
> of
> the class. So:
>
> struct foo; //incomplete
> void foo::do_something();
>
> would be semantically the same as:
>
> struct foo; //incomplete
> void do_something( foo * );
>
> the only difference being in syntax: the former would be callable using
> the
> dot syntax.

Uh, understand. So, it's actually member SYNTAX for non-member
functions. Exactly as in the proposal that Peter sited. So, how the
"non-friend" qualification's got into the discussion? It's not in the
proposal. Friend or no-friend has no difference, right?

That brings me back to the original Robert's question about
accessibility scope that you addressed with "friend"... Something that I
feel is "restoring data+behavior association after breaking it".

[Vladimir Batov]

On 2016-06-01 11:57, Howard Hinnant wrote:
> On May 31, 2016, at 9:51 PM, Vladimir Batov
>>

>> From efficiency point of view the above is identical to
>>
>> Book::~Book() {}
>>
>> or compiler writers do more magic with "=default"? I like "=default"
>> as I feel it's cleaner language-wise. What weighty reason can I give
>> to a beginner to prefer "=default" over {}... apart from "I am older,
>> musclier and your boss" :-)
>
> That’s a good question. It doesn’t turn the destructor from
> non-trivial to trivial. It does not change the noexcept or constexpr
> status. The only thing it really does is say: I want default
> semantics here. Maintenance programmers should be more leery of
> adding superfluous stuff to your destructor.

Good point. Indeed, it's just too easy/tempting to stick something
between the braces. :-)

[Vladimir Batov]

On 2016-06-01 11:40, Gavin Lambert wrote:
> On 1/06/2016 12:35, Vladimir Batov wrote:
>> I am not sure if "to allow free function invocation syntax to invoke
>> member functions and vice-versa" is exactly what Emil wanted. Quoting
>> from the top --
>> the "declaration of non-friend "member" functions outside of the type
>> definition". I read it as Emil wants it to be a "member" but declared
>> "outside".
>
> That's what extension methods are though, as I mentioned in my other
> post.

Even though Emil did clarify that he indeed meant "extension methods"
functionality, I really feel it begs for a correction as I feel we are
dangerously mixing the terms. Those "extension methods" are not members.
All the proposal suggests is to allow member syntax (the emphasis on
syntax) for free-standing functions... friends or no friends. "Member
syntax" to "member" is like Elvis' cuff-link to Elvis. :-) Big
difference... IMO.

> ...

I snipped the rest as I do agree with your point... which is not a bad
thing, right? :-)

[Vladimir Batov]

On 2016-06-01 11:30, charleyb123 . wrote:
> ...

> In short, what I like:
>
> *- some kind of "std_pimpl<>" provides a standardized interface, and
> "plumbing"/implementation.
> *- CRTP in the "std_impl<>" allows for a zero-cost compile-time
> implementation-override.
> *- If necessary, implementation can permit "std_pimpl<>" to be
> parameterized with merely a "Foo" declaration.

Seems like the proposed pimpl has it all from the list. If not, could
you please give an example of what you want but the proposed pimpl falls
short. I'll try and see if I can accommodate your case.

[Emil Dotchevski]

On Tue, May 31, 2016 at 7:28 PM, Vladimir Batov <
Vladimi...@constrainttec.com> wrote:

> On 2016-06-01 11:40, Gavin Lambert wrote:
>
>> On 1/06/2016 12:35, Vladimir Batov wrote:
>>
>>> I am not sure if "to allow free function invocation syntax to invoke
>>> member functions and vice-versa" is exactly what Emil wanted. Quoting
>>> from the top --
>>> the "declaration of non-friend "member" functions outside of the type
>>> definition". I read it as Emil wants it to be a "member" but declared
>>> "outside".
>>>
>>
>> That's what extension methods are though, as I mentioned in my other post.
>>
>
> Even though Emil did clarify that he indeed meant "extension methods"
> functionality, I really feel it begs for a correction as I feel we are
> dangerously mixing the terms.

Yes I stand corrected. Different syntax, same idea.

I still don't think that it's that important of a feature, since my premise
is that there are no benefits to the member function call syntax.

Emil

[Rob Stewart]

On May 31, 2016 7:09:22 PM EDT, Emil Dotchevski <emildot...@gmail.com> wrote:
>On Tue, May 31, 2016 at 4:05 PM, rstewart <rste...@ptd.net> wrote:
>
>> "Emil Dotchevski" <emildot...@gmail.com> wrote:
>> > On Tue, May 31, 2016 at 2:48 PM, Rob Stewart <rste...@ptd.net>
>wrote:
>> > > On May 31, 2016 5:41:54 PM EDT, Emil Dotchevski <
>> emildot...@gmail.com>
>> > > wrote:
>> > > >On Tue, May 31, 2016 at 2:29 PM, Chris Glover
><c.d.g...@gmail.com>
>> > > >wrote:
>> > > >
>> > > >I'm generally not in favor of adding stuff to C++. What's the
>>upside in
>> > > >this case? To be able to say p.do_something() instead of
>> > > >do_something(p), because the latter offends Java programmers? :)

>> > >
>> > > The upside is not writing some calls one way and others the other
>>way on
>> > > the same object, and having to remember which is which.
>> > >
>> > So, don't use the dot syntax. :)
>>
>> Not all functions can be non-members.
>

>Do you mean e.g. virtual functions? They can be hidden behind free
>function wrappers.

I had in mind operators like +=, for example.

___
Rob

(Sent from my portable computation engine)

[Rob Stewart]

On May 31, 2016 7:47:25 PM EDT, Gavin Lambert <gav...@compacsort.com> wrote:
>On 1/06/2016 11:05, rstewart wrote:
>>>> The upside is not writing some calls one way and others the other
>way on
>>>> the same object, and having to remember which is which.
>>>>
>>> So, don't use the dot syntax. :)
>>
>> Not all functions can be non-members.
>
>Operators can be;

Not all operators can be: consider +=, -=, *=, /=, ->, and &.

___
Rob

(Sent from my portable computation engine)

[dariomt]

Peter Dimov <lists <at> pdimov.com> writes:
> Howard Hinnant wrote:
> > It has to be outlined, in the source.cpp:
> >

> > class Book
> > {
> > struct implementation;
> > std::unique_ptr<implementation> impl_;
> > public:
> > ~Book();
> > Book();
> > };
>
> You could probably do
>
> // hpp
> class Book
> {
> struct implementation;
> std::unique_ptr<implementation, void(*)(implementation*)> impl_;
> public:
> Book();
> };
>
> // cpp
> Book::Book(): impl_( new implementation,
checked_delete<implementation> ) {}
>

Sorry if this is a silly question, but wouldn't this double the size of
member variable impl_ (two pointers), just to save some typing?

To me it doesn't look like a good tradeoff.

I'm quietly following this thread, and I still don't get why
std::unique_ptr is not good enough for heap-allocated pimpl.

[Rob Stewart]

On May 31, 2016 7:13:53 PM EDT, Emil Dotchevski <emildot...@gmail.com> wrote:

>On Tue, May 31, 2016 at 4:07 PM, rstewart <rste...@ptd.net> wrote:
>> "Emil Dotchevski" <emildot...@gmail.com> wrote:

>> > On Tue, May 31, 2016 at 2:45 PM, Rob Stewart <rste...@ptd.net>
>wrote:
>> > > On May 31, 2016 5:10:35 PM EDT, Emil Dotchevski <
>> emildot...@gmail.com>
>> > > wrote:
>> > > >
>> > > >header:
>> > > >
>> > > >struct foo { int critical_; };
>> > > >shared_ptr<foo> create_foo();
>> > > >inline void use_foo_critical( foo * p ) { ++p->critical_; }
>> > > >void use_foo_not_so_critical( foo * )
>> > >
>> > > Were you leaving encapsulation as an exercise for the reader?
>> > >
>> > > class foo
>> > > {
>> > > public:
>> > > use_critical () { ++critical_; }
>> > > protected:
>> > > int critical_;
>> > > };
>> > >
>> > > Your foo_ can still access critical_.
>> >
>> > Right, though my preference is as follows.
>> >
>> > //Public interface:
>> >
>> > namespace boost { template <class> class shared_ptr; }

You can't do the same thing with std::shared_ptr. Besides, the user of your foo will need the definition of the same shared_ptr as does create_foo(), do you save anything by avoiding the include directive?

>> > class foo;
>> > boost::shared_ptr<foo> create_foo();
>> > void use_foo_critical( foo * );
>> > void use_foo_not_so_critical( foo * );
>> >
>> > //Implementation details:
>> >
>> > class foo
>> > {
>> > foo( foo const & );
>> > foo & operator=( foo const & );
>> > int critical_;
>> > friend void use_foo_critical( foo * );
>> > protected:
>> > foo();
>> > ~foo();
>> > };
>> > inline void use_foo_critical( foo * p ) { ++p->critical_; }
>>
>> That doesn't provide inline access to those with access only to the
>header.
>
>I meant that whole thing being in the header, which is required for
>inline
>access (within the CPP file inline is next to pointless since the
>compiler
>can easily inline any function.) The point I'm making with
>"//implementation details" is that the fact that use_foo_critical is
>defined inline, as well as the exact definition of foo itself, is not
>part
>of the interface and thus subject to change.

As you guessed, I assumed that "Implementation details" referred to code in a separate source file.

Unlike a detail namespace, a mere comment separates users from what you consider implementation details. The compiler can't help you, though I suppose you can implement, in the header, only what needs exposure for performance reasons.

___
Rob

(Sent from my portable computation engine)

[Rob Stewart]

On May 31, 2016 5:56:29 PM EDT, Josh Juran <jju...@gmail.com> wrote:
>On May 31, 2016, at 5:10 PM, Emil Dotchevski <emildot...@gmail.com>
>wrote:
>
>> On Tue, May 31, 2016 at 12:09 PM, Chris Glover <c.d.g...@gmail.com>
>wrote:
>>
>>> I sometimes don't pimpl an entire object. This might be because I
>have
>>> templates in the interface or because I need the functions to
>inline. Your
>>> solution doesn't allow me to do that.
>>
>> One possibility is to use inheritance in the cpp file:

>>
>> header:
>>
>> struct foo { int critical_; };
>>

>> cpp:
>>
>> namespace
>> {
>> struct foo_: foo
>> {
>> int not_so_critiral_;
>> };
>> }
>
>Since this uses a derived class instead of a pointer, I've been calling
>this the "dimpl" idiom. :-)

I like the name. It's a nice play on Herb's "Pimpl".

[Domagoj Saric]

On 28.5.2016. 7:41, Vladimir Batov wrote:
<...>
> Thoughts? No pressure. I am easy either way. :-)

* +1 for 'standardizing' pimpling (and bringing it up yet again;)


* -1 for 'breaking namespace usage' (forcing implementation in the
global namespace)
This is IMO not 'a matter of style', you could otherwise push it a
little more and say that C++ is a 'specific style of C'...

There are several different ways to address this,
https://github.com/psiha/pimpl/tree/master/include/boost/pimpl:

1) auto_pimpl.hpp ('public'/interface side include), ln 85: impl()
member function w/ C++14 return type deduction

2) auto_impl.hpp ('private'/impl side include), ln 48:
struct implementation "Interface -> implementation mapping metafunction"

So:
a) by default impl() 'asks' 'boost::pimpl::implementation<>' for the
impl-type to which it needs to cast the this pointer
b) by default implementation<Interface>::type maps to Interface::impl.
...and each of those steps can be customized/specialized...

So, by default, if you have a class my_interface, you would only need to
include a forward declaration for impl in the interface:
class my_interface { public: class impl; }
and then implement it in the .cpp.

If that isn't satisfactory one can specialize:
- boost::pimpl::implementation<my_class> (to 'point' to a different type)
>or<
- boost::pimpl::*_object<my_class>::impl() (not to use the default
implementation<> metafunction)

This could, as always, be further refined 'by yet another layer of
indirection', by using a free function for the 'Interface & -> Impl &'
conversion (which could then simply overloaded in the user's namespace
and have ADL kick in)...


Considering I'm always for giving more options (i.e. the opposite of the
shared_ptr approach), and how differing opinions obviously still are WRT
pimpling, I'd vote that at least the boost (if not the eventual std)
pimpl version provides as many customization points as possible until
usage patterns 'fall into place'...


* -1 for forcing heap usage
https://github.com/psiha/pimpl/blob/master/README.md
This is a complete C++14 rewrite of an old private pimpl framework which
features a 'stack' or 'auto' (as in automatic storage) pimpl and as well
as a heap based version. This is very recent undertaking, the heap
version hasn't been ported at all, and the auto version doesn't yet
handle the problematic use cases, inheritance and polymorphism (which
are much more easily handled in the heap version)...

And no, allocators won't ('fully') do it. You still have extra
indirection and even more code complexity (allocators which use stack in
the current function? ugh...)...


* -1 for the 'pointer/value semantics' 'kludge'
IMO this is completely misplaced, I can't see it serving as anything
else but replacing/wrapping the standard arrow/-> (smart)pointer syntax
with the dot syntax. If the particular class objects are always to be
heap allocated simply use them through a pointer...


* -1 for paying for shared_ptr for inherently shared classes of objects
Use boost::intrusive_ptr for those (in fact make the 'actual' factory
function, that is implemented on the .cpp side, private and have it
return a naked pointer, so that it can return it through a register
across the ABI, and then wrap on the interface side in an inline member
that returns a smart_ptr)...


--
"What Huxley teaches is that in the age of advanced technology,
spiritual devastation is more likely to come from an enemy with a
smiling face than from one whose countenance exudes suspicion and hate."
Neil Postman

[charleyb123 .]

On Tue, May 31, 2016 at 9:36 PM, Vladimir Batov <
Vladimi...@constrainttec.com> wrote:

> On 2016-06-01 11:30, charleyb123 . wrote:
>
>> ...
>> In short, what I like:
>>
>> *- some kind of "std_pimpl<>" provides a standardized interface, and
>> "plumbing"/implementation.
>> *- CRTP in the "std_impl<>" allows for a zero-cost compile-time
>> implementation-override.
>> *- If necessary, implementation can permit "std_pimpl<>" to be
>> parameterized with merely a "Foo" declaration.
>>
>
> Seems like the proposed pimpl has it all from the list. If not, could you
> please give an example of what you want but the proposed pimpl falls short.
> I'll try and see if I can accommodate your case.

This approach uses no namespaces.

Overrides can be free-functions, or fully scoped as members of
application-specific types, with no possibility of ADL intercepting the
intended implementation.

--charley

[Vladimir Batov]

On 2016-06-01 20:40, Domagoj Saric wrote:
>
> * +1 for 'standardizing' pimpling (and bringing it up yet again;)

Good.

> * -1 for 'breaking namespace usage' (forcing implementation in the

> global namespace) ...

I believe that's been addressed... From the user perspectives anyway.
Your mileage may vary.

> * -1 for forcing heap usage...

This is incorrect. The proposed design is policy-based, i.e.
configurable/customizeable. Indeed, I personally only implemented
heap-based policies that I care about. Care to provide a stack-based
policy based on your implementation? That already was one the requests
on the thread.

> And no, allocators won't ('fully') do it. You still have extra
> indirection and even more code complexity (allocators which use stack
> in the current function? ugh...)...

I would not be so rush dismissing allocators. There must be reasons why
we have them in the Standard.

> * -1 for the 'pointer/value semantics' 'kludge'

Not sure what you mean by that. Anyway, I'll provide some history. As
you are aware that pimpl stands for "pointer to implementation". Stress
is on "pointer". Kludge or not from the set-go very early
implementations of the idiom had pointer-semantics. In fact, they *were*
pointers (opaque pointers throughout, say, X Window System). Then, when
we had a pimpl-related discussion on the list many moons ago "value
semantics" were requested and been provided. Now it's implemented as a
manager. If you do not need/like it, you simply do not use it. You
provide your own stack-based manager (incidentally with value-semantics)
and use it. Does it sound acceptable?

> IMO this is completely misplaced, I can't see it serving as anything
> else but replacing/wrapping the standard arrow/-> (smart)pointer
> syntax with the dot syntax. If the particular class objects are always
> to be heap allocated simply use them through a pointer...

I am under impression you've disposed of the pivotal pimpl property --
implementation hiding. Looking at your implementation seems to confirm
that view... I could be wrong. If my understanding and reading of your
code are correct, then, yes, indeed with no implementation-hiding goal
all that code is "completely misplaced" and "serving as anything else
but" and unnecessary. I agree.

> * -1 for paying for shared_ptr for inherently shared classes of objects
> Use boost::intrusive_ptr for those (in fact make the 'actual' factory
> function, that is implemented on the .cpp side, private and have it
> return a naked pointer, so that it can return it through a register
> across the ABI, and then wrap on the interface side in an inline
> member that returns a smart_ptr)...

You might be right and boost::intrusive_ptr is far superior to
std::shared_ptr. I personally never liked the intrusiveness of
boost::intrusive_ptr. And I suspect that that requirement of
boost::intrusive_ptr won't sit well with at least some users as well.

As the intrusive_ptr docs state: "As a general rule, if it isn't obvious
whether intrusive_ptr better fits your needs than shared_ptr, try a
shared_ptr-based design first". And it is not obvious to me at the
moment.

I believe shared_ptr is as efficient as boost::intrusive_ptr when
created with make_shared or allocate_shared (used in the proposal).
Well, apart from the 2-pointers' foot-print... which IMO is the proper
way to go.

[Domagoj Saric]

On 1.6.2016. 14:05, Vladimir Batov wrote:
>> * -1 for 'breaking namespace usage' (forcing implementation in the
>> global namespace) ...
>
> I believe that's been addressed... From the user perspectives anyway.
> Your mileage may vary.

Could you please point me to the post(s) that address this? AFAICT
you've mostly just dismissed the criticisms as subjective (which I
described as an extreme position WRT what can be named a matter of
style, especially when more conventional 'styles', that do not pollute
the global namespace, have been presented)...

>> * -1 for forcing heap usage...
>
> This is incorrect. The proposed design is policy-based, i.e.
> configurable/customizeable. Indeed, I personally only implemented
> heap-based policies that I care about. Care to provide a stack-based
> policy based on your implementation? That already was one the requests
> on the thread.

OK, I'm sorry, I somehow missed this possibility. I didn't see it
offered and the code seemed a bit convoluted to deduce how configurable
it actually is - a major culprit being the fact that you've made the
individual policies/'managers' members of the master class template.
These should be separate, decoupled classes that can live in separate
headers and be included only when desired...

>> And no, allocators won't ('fully') do it. You still have extra
>> indirection and even more code complexity (allocators which use stack
>> in the current function? ugh...)...
>
> I would not be so rush dismissing allocators. There must be reasons why
> we have them in the Standard.

Whatever the reasons why we have them, the points raised still hold...

>> * -1 for the 'pointer/value semantics' 'kludge'
>
> Not sure what you mean by that. Anyway, I'll provide some history. As
> you are aware that pimpl stands for "pointer to implementation". Stress
> is on "pointer". Kludge or not from the set-go very early
> implementations of the idiom had pointer-semantics. In fact, they *were*
> pointers (opaque pointers throughout, say, X Window System). Then, when
> we had a pimpl-related discussion on the list many moons ago "value
> semantics" were requested and been provided. Now it's implemented as a
> manager. If you do not need/like it, you simply do not use it. You
> provide your own stack-based manager (incidentally with value-semantics)
> and use it. Does it sound acceptable?

No :)

The opaque pointers were/are used as pointers, not through adapters that
made them look like references.

I'm aware of the history and how, once you already had an API used by
other code, the 'easiest' way to hide the implementation was to 'store'
it in a pointer and forward all the calls (requiring little or no change
in 'client' code). However, just because that's the 'historical
evolutionary path' or that it maps to 'pimpl', a 'cool tongue-in-cheek
name' isn't an argument that that's the way it should be (the 'is-ought'
distinction):

* Supporting automatic-storage allocation automatically 'already'
doesn't square with the pimpl moniker. I'd rather lose or 'misuse' the
established idiom name than efficiency (and simpler user code).

* 'Pointer-semantics'/'shallow-copy' types are..um...'unusual' (like
having a pointer with the dot syntax).

* If all of a type's implementation is hidden in a pointer-to-impl it
means it is always dynamically allocated so just make it so. Instead of
class interface
{
public:
void foo();
private:
impl * pimpl;
};

just do

class interface
{
public:
void foo();

static smart_ptr<interface> create();

protected:
interface() = default;
~interface();
};

// in .cpp

class impl : interface
{ ... };

void interface::foo()
{
static_cast<impl &>( *this )....
}

...this is also slightly more efficient as you eliminate one pointer
indirection...

>> IMO this is completely misplaced, I can't see it serving as anything
>> else but replacing/wrapping the standard arrow/-> (smart)pointer
>> syntax with the dot syntax. If the particular class objects are always
>> to be heap allocated simply use them through a pointer...
>
> I am under impression you've disposed of the pivotal pimpl property --
> implementation hiding. Looking at your implementation seems to confirm
> that view... I could be wrong. If my understanding and reading of your
> code are correct, then, yes, indeed with no implementation-hiding goal
> all that code is "completely misplaced" and "serving as anything else
> but" and unnecessary. I agree.

Even if my code is convoluted it still a priori cannot be the way you
understood it - what would it be for then (a pimpl that doesn't pimpl
the impl)? :)

My 'pimpl impl' does hide the implementation, in fact it hides even the
pimpl's private/implementation side parts from the interface/client side
(that's why it has two headers, pimpl.hpp included on the interface side
and impl.hpp included on the implementation side) unlike you
implementation that has everything, and all the policies/managers, in
one header...

>> * -1 for paying for shared_ptr for inherently shared classes of objects
>> Use boost::intrusive_ptr for those (in fact make the 'actual' factory
>> function, that is implemented on the .cpp side, private and have it
>> return a naked pointer, so that it can return it through a register
>> across the ABI, and then wrap on the interface side in an inline
>> member that returns a smart_ptr)...
>
> You might be right and boost::intrusive_ptr is far superior to
> std::shared_ptr.

It is different, not necessarily superior.

> I personally never liked the intrusiveness of
> boost::intrusive_ptr.

That's kind of like saying I never liked the sharedness of shared_ptr :)

> And I suspect that that requirement of
> boost::intrusive_ptr won't sit well with at least some users as well.

a) It is/would be an implementation detail?
b) Why?

> As the intrusive_ptr docs state: "As a general rule, if it isn't obvious
> whether intrusive_ptr better fits your needs than shared_ptr, try a
> shared_ptr-based design first".

That just follows from the 'managed' ('configurability discourages use')
mindset behind the whole smart_ptr library.

> And it is not obvious to me at the moment.

If a type is always allocated on the heap and inherently has shared
ownership semantics (e.g. any type deriving from
enable_shared_from_this) then 'obviously' reference counting has already
'intruded' it 'by definition'...

> I believe shared_ptr is as efficient as boost::intrusive_ptr when
> created with make_shared or allocate_shared (used in the proposal).

Instead of believing (WRT 'things of this world') you may be better
served by checking the evidence (e.g. codegen, binary size etc.).
intrusive_ptr won't force an indirect deleter, RTTI, aditional pointer
and a 'control object' 'on you'...

> Well, apart from the 2-pointers' foot-print... which IMO is the proper
> way to go.

Why is that the way to go?

--
"What Huxley teaches is that in the age of advanced technology,
spiritual devastation is more likely to come from an enemy with a
smiling face than from one whose countenance exudes suspicion and hate."
Neil Postman

[Vladimir Batov]

Domagoj,

Apologies for top posting. I started writing a point-by-point reply just
to realize what it was unlikely to have any impact. The way we view the
pimpl idiom, its deployment, applicability scope, acceptable design and
overhead are drastically different. You do not seem to like the proposed
implementation of "policies", you do not like allocators, you do not
like pointer/value semantics, you do not like deployment of
std::shared_ptr. And that's fine.

I had another look at your implementation and indeed it does seem to
hide the implementation. Apologies. It does not seem to provide or to be
extendible to support pointer-semantics or polymorphic hierarchies or
variable-size structures/records which you seem to dismiss but which are
important to me. Therefore, in order to move forward, one possibility is
that In addition to the existing managers you write an additional
policy/manager as you see it with no allocators, shared_ptr, etc. as
part of the proposed policy-based extendible framework. That way people
will have a choice of policies which is a good thing IMO as the end
product will be useful for us all.

Alternatively, you might like to explore the possibility of proposing
your design, say, on this list. Your design might well be better and
more successful. IMO any outcome is good as I think it'd be useful to
have a pimpl entry in Boost.

Does it sound reasonable?

[Andrzej Krzemienski]

Hi Vladimir,
I am following the discussion about Pimpl here, and it seems to me I
understand the idea less and less.

My impression is (but I might be wrong here) that people use Pimp for
slightly different things and that one solution might not suit all the
needs. Let me list two use cases that I consider significantly different:

1a. I am selling my own complicated library, and I want to make sure users
will never see the source code. I can afford to add some run-time penalty
because this is a high-level library and the cost of additional indirection
is negligible compared to what the library is doing.

1b. I am a programmer and I use a third party library called LIB1 but I
wrap it into a class C because I do not want the users of class C to have
any knowledge of LIB1, because next month I want to replace library LIB1
with another library LIB2, and I want to do it seamlessly, without forcing
the users of C to recompile anything. Again, the cost of additional
indirection is negligible, because what LIB1 and LIB2 do is orders of
magnitude more expensive.

2. I want to apply Pimpl to every second class in my program because my
code is not performance-critical and I would rather have fast compile-times
than fast run-times. A valid trade-off for non-critical parts of the
program.

The question is, which of the two (or any other) use cases does your
library address?

Also, I remember that last time the library promoted the use of shared
ownership (altering the state of one object immediately alters the states
of other objects). The examples from the new dos still seem to imply that.
Is it necessary? If for some reason I need the shared ownership I could
implement it atop of Pimpl classes using a shared_ptr.

But I might be wrong about my understanding of the library. I am really
missing the introduction, that would outline the scope of the problems you
are addressing, some high level design decisions, and the intended usage.

For now, the initial example encourages me to use shared semantics, and I
feel it is encouraging me to do the wrong thing.

Regards,
&rzej

2016-05-28 7:41 GMT+02:00 Vladimir Batov <Vladimi...@constrainttec.com>:

> Don't laugh but it is only fairly recently at work that I managed to move
> all our supported platforms to C++11. So, then, I started moving our code
> to C++11 (what a delight!) and stumbled upon my old pimpl which is used
> quite a bit around my workplace for all its good properties... :-)
>
> Now C++11-ed piml turned out to be very small and very basic. So basic it
> felt it did not have the right to exist. Still, it encapsulates and
> enforces the "proper" pimpl properties and behavior, reduces implementation
> minutia and offers a recognizable deployment pattern (helps other people
> reading the code).
>
> Over the years IMO the technique has been proven as legitimate and useful,
> a basic but important building block... rather than a curiosity item.
> Unfortunately, despite many articles and a few Sutter's GotWs about it
> there is nothing ready-to-go like std::unique_ptr.
>
> I feel that pimpl and its deployment pattern need to be codified,
> described, "standardized" and in our toolboxes to stop/avoid everyone
> re-inventing the pimpl and making the same mistakes. IMO Boost is best
> positioned for that.
>
> Do you think we might review what I've got and/or maybe collectively come
> up with something better... It's no MPL or Hana but as std::unique_ptr it's
> one of the first "little things" I personally reach out for in my everyday
> work. IMO having pimpl in Boost would save quite a few hours of frustration
> for many.

>
> Thoughts? No pressure. I am easy either way. :-)
>

> https://github.com/yet-another-user/pimpl
>
> Docs are freshened up but still somewhat out of date. Apologies.

[Vladimir Batov]

Andrzej,

Thank you for your input. It's appreciated. Please find my replies
below.

On 2016-06-06 18:45, Andrzej Krzemienski wrote:
> Hi Vladimir,
> I am following the discussion about Pimpl here, and it seems to me I
> understand the idea less and less.

In my view the idea has not changed at all. Not even a little. As I see
it the pimpl idea is still 1) to separate interface from the
implementation and 2) to hide the implementation.

And I personally did not see this thread diverging from these two
principal pimpl properties. IMO the discussion revolved around the
implementation. Say, Emil (as I understand) was advocating taking the
interface separation part to the "extreme" -- eliminating the
interface/proxy object altogether and providing the interface via public
functions. Domagoj suggested his own alternative implementation also.

> My impression is (but I might be wrong here) that people use Pimp for
> slightly different things and that one solution might not suit all the
> needs.

Well, I feel that people use pimpl to do #1 (separate) and #2 (hide). If
they use pimpl to do something else, then they might be trying the wrong
tool.

Still, people requirements and priorities are different so, indeed, "one
solution might not suit all the needs". That's why the proposal has, in
fact, two solutions -- with pointer and value-semantics. More so, this
time the proposal is policy-based and extendible. I do not claim to
cover every-man requirements and needs. I feel, the design is extendible
and sufficiently flexible to accommodate new requirements and needs.
That is why I am trying to encourage Domagoj to incorporate his
stack-based policy/manager. That way we'd cover all pimpl
requirements/expectations that I am aware of so far.

All three cases seem to be the standard cases for pimpl idiom
application as in every case you want separate+hide. I believe the
proposed pimpl handles all three. You can deploy pimpl<Foo>::shared or
pimpl<Foo>>::unique depending on your circumstances.

Better still, if, say, Domagoj decides to contribute his stack-based
policy implementation, then, say, pimpl<Foo>::stack might be deployed in
applications with stricter efficiency/memory requirements.

> Also, I remember that last time the library promoted the use of shared
> ownership (altering the state of one object immediately alters the
> states
> of other objects). The examples from the new dos still seem to imply
> that.
> Is it necessary? If for some reason I need the shared ownership I could
> implement it atop of Pimpl classes using a shared_ptr.

Yes, indeed, I think it is probably a valid point. The thing is I
initially wrote it for myself. Probably due to my application-domain
specifics I still use pimpl::shared exclusively. So, for me value-pimpl
wrapped in shared_ptr seemed like unnecessary inconvenient complexity.
If someone else with different priorities decides to submit his
value-semantics pimpl and it gets accepted, I'll use it happily as you
describe.

> But I might be wrong about my understanding of the library. I am really
> missing the introduction, that would outline the scope of the problems
> you
> are addressing, some high level design decisions, and the intended
> usage.

I tend to agree with you and "some high level design decisions, and the
intended usage" would be beneficial as I feel pimpl can be easily
mis-used. I would most likely go down that road if pimpl gets into
Boost. As it is now I am far from certain if I can justify any
additional effort documentation-wise.

> For now, the initial example encourages me to use shared semantics, and
> I
> feel it is encouraging me to do the wrong thing.

I am not sure I can agree with that. The proposal has two behavioral
facets -- pointer-semantics and value-semantics. I had to start the
description with one or the other. I happened to choose the
pointer-semantics as it's easier to describe as it is shared_ptr-based.
After that, again, value-semantics implementation is easier to describe
as its underlying implementation mimics shared_ptr's
incomplete-type-management functionality. Still, I think you are right
sensing that I feel more for pointer-semantics :-) but I would not
interpret it as an encouragement to do the wrong thing. :-)

Again, I do agree that the documentation needs more "do it", "don't do
it", "best practices", etc. I just do not feel like investing more of my
time due to high probability of it all wasted.

[Rainer Deyke]

On 06.06.2016 10:45, Andrzej Krzemienski wrote:
> Hi Vladimir,
> I am following the discussion about Pimpl here, and it seems to me I
> understand the idea less and less.
>
> My impression is (but I might be wrong here) that people use Pimp for
> slightly different things and that one solution might not suit all the
> needs. Let me list two use cases that I consider significantly different:
>
> 1a. I am selling my own complicated library, and I want to make sure users
> will never see the source code. I can afford to add some run-time penalty
> because this is a high-level library and the cost of additional indirection
> is negligible compared to what the library is doing.
>
> 1b. I am a programmer and I use a third party library called LIB1 but I
> wrap it into a class C because I do not want the users of class C to have
> any knowledge of LIB1, because next month I want to replace library LIB1
> with another library LIB2, and I want to do it seamlessly, without forcing
> the users of C to recompile anything. Again, the cost of additional
> indirection is negligible, because what LIB1 and LIB2 do is orders of
> magnitude more expensive.
>
> 2. I want to apply Pimpl to every second class in my program because my
> code is not performance-critical and I would rather have fast compile-times
> than fast run-times. A valid trade-off for non-critical parts of the
> program.
>
> The question is, which of the two (or any other) use cases does your
> library address?

Personally, when I have used pimpl, it has always been for another use
case entirely:

3. I want a class to have value semantics, but use a copy-on-write
implementation behind the scenes. This is mostly for objects that get
copied around a lot. I don't particularly care about implementation
hiding or compile times. I care about memory usage and raw performance.
Pimpl gives me that for copy operations, at the cost of an extra level
of redirection on other operations.


--
Rainer Deyke (rai...@eldwood.com)

[Emil Dotchevski]

On Wed, Jun 8, 2016 at 2:29 PM Rainer Deyke <rai...@eldwood.com> wrote:

> 3. I want a class to have value semantics, but use a copy-on-write
> implementation behind the scenes.

Pimpl is not about semantics, it's about reducing physical coupling. It's
not about the pointer pointing at the private implementation object (which,
being private, is an implementation detail anyway), it's about the private
implementation *type* being left incomplete.

Emil

[Rainer Deyke]

On 08.06.2016 23:57, Emil Dotchevski wrote:
> On Wed, Jun 8, 2016 at 2:29 PM Rainer Deyke <rai...@eldwood.com> wrote:
>
>> 3. I want a class to have value semantics, but use a copy-on-write
>> implementation behind the scenes.
>
> Pimpl is not about semantics, it's about reducing physical coupling. It's
> not about the pointer pointing at the private implementation object (which,
> being private, is an implementation detail anyway), it's about the private
> implementation *type* being left incomplete.

Pimpl is a technique where the implementation of a class is moved to a
separate implementation class (the "impl"), to which the interface class
maintains a pointer (the "p"). One advantage of this technique is that
it decreases physical coupling. Another advantage of this technique is
that it allows the copy-on-write optimization (which is an
implementation detail of the interface class, but a potentially
important one).

Are arguing that my use of pimpl isn't true pimpl? If so, I disagree.
I use all aspects of the pimpl idiom. I receive all of the benefits of
the idiom and pay all of the costs. That I just happen to value what is
generally perceived as at best a secondary benefit (i.e. the possibility
of copy-on-write) over what is generally perceived as the primary
benefit (i.e. decreasing physical coupling) does not change that.


--
Rainer Deyke (rai...@eldwood.com)

[Emil Dotchevski]

On Thu, Jun 9, 2016 at 5:19 AM, Rainer Deyke <rai...@eldwood.com> wrote:

> On 08.06.2016 23:57, Emil Dotchevski wrote:
>
>> On Wed, Jun 8, 2016 at 2:29 PM Rainer Deyke <rai...@eldwood.com> wrote:
>>
>> 3. I want a class to have value semantics, but use a copy-on-write
>>> implementation behind the scenes.
>>>
>>
>> Pimpl is not about semantics, it's about reducing physical coupling. It's
>> not about the pointer pointing at the private implementation object
>> (which,
>> being private, is an implementation detail anyway), it's about the private
>> implementation *type* being left incomplete.
>>
>
> Pimpl is a technique where the implementation of a class is moved to a
> separate implementation class (the "impl"), to which the interface class
> maintains a pointer (the "p").

The "p" stands for "private" not for pointer. The defining property of the
pimpl idiom is that the private implementation type is left incomplete in
the interface: the pointer is opaque. For example, if you implement
copy-on-write using a pointer to a type that is defined in the interface,
you're not using pimpl. So, logical design (semantics), including copy
semantics, is orthogonal to pimpl.

Emil

[Vladimir Batov]

Emil, as Sutter described in his early articles the pimpl name was
introduced by one of his colleagues where the "p" stood for "pointer"
due to Hungarian notation as the type was actually a pointer to "impl".

That said, I agree with you stressing the "private implementation"
property. However, IMO that property only comes after the "separation of
interface and implementation". So, one might argue that a class
implementing the "separation" property already qualifies as a pimpl (or
Handle or Proxy)... and formally I personally agree that it does. The
first (and one might argue the only) pimpl property is the "separation",
the introduction of a level of indirection, the introduction of a proxy.

Many use it to then hide the implementation. Rainer uses it for
write-on-copy. I personally find Rainer's application of Pimpl
interesting (and I am considering adding such a policy to the proposed
pimpl).

Then, when you say that "Pimpl is *not* about semantics", I want to
agree with you... but in reality I have to disagree. :-) The reason for
that is that (for me) the idiom is about indirection -- through handle
to body, through proxy to data, i.e. the idiom defines *two* objects...
Even your version has two objects -- opaque data and shared_ptr<data>.

And, as soon as we introduce 2 objects, we face the important dilemma of
defining the relationship between those two objects. The semantics
describes that relationship -- be that the pointer semantics, the value
semantics or, as Rainer mentioned, the copy-on-write semantics. One
might argue that that's outside the idiom. However, I feel that's
splitting a hear as on the practical level the proxy-data relationship
is part-n-parcel of the whole package. Don't you agree?

[Vladimir Batov]

On 06/10/2016 04:59 AM, Emil Dotchevski wrote:

>> ... The defining property of the

>> pimpl idiom is that the private implementation type is left
>> incomplete in

>> the interface: the pointer is opaque...
>>
>> Emil

Just happened to re-read Gamma's "Design Patterns". The Bridge
section... which is the Pimpl...

There the "hide the implementation" is mentioned in the Applicability
section, i.e. what the pattern is good for... i.e. it's not the
pattern's property... it's "only" its applicability... in the context of
C++.

As for copy-on-write, then Gamma et al explicitly mention it... in the
Proxy pattern... which they position separately from Bridge... they do
not even mention it in the Related Section... which to me is a bit too
subtle... but admittedly subtlety is not my strong suit.

[Rainer Deyke]

On 09.06.2016 20:59, Emil Dotchevski wrote:
> The "p" stands for "private" not for pointer. The defining property of the
> pimpl idiom is that the private implementation type is left incomplete in
> the interface: the pointer is opaque.

I do, of course, define my implementation class in a .cpp file, with
only a forward declaration in the .hpp file. Since I'm paying the price
of pimpl, I might as well get the full benefit.


--
Rainer Deyke (rai...@eldwood.com)