Proposal: Private extension methods

[fmatth...@gmail.com]

Hello everyone, I am going to submit a proposal for relaxing the restriction on requiring private non-virtual class methods and static private class methods to be declared in the class definition.

The github repository with the current draft of the proposal is here:

https://github.com/fmatthew5876/stdcxx-privext

The proposal came out of the discussion from this thread:

https://groups.google.com/a/isocpp.org/forum/#!searchin/std-proposals/private/std-proposals/xukd1mgd21I/akgZSTG4NBUJ

[Andrew Tomazos]

Hi Matthew,

I did briefly read your proposal.

You touch on this a little in your proposal, but just to be clear 9.4p5 [class.static]:

> When used in the declaration of a class member, the static specifier shall only be used in the member declarations that appear within the member-specification of the class definition.

So this is currently ill-formed:

struct X

{

    void f();

};

static void X::f() {} // ERROR

This frees the static specifier up to be used to specify a static member function if that is what you want.  There is no ambiguity with the internal linkage meaning on a free function.  In the event someone accidentally places static in front of a private extension method, intending internal linkage but getting a static member function, then the only effect is a compile-time error when they try to access the implicit object parameter that doesn't exist.  It isn't dangerous.

You might also want to consider giving private extension methods implicit internal linkage (both static and non-static member functions), the only use case I can imagine is placing them in the .cpp so the linker doesn't even need to see the symbol, and if you really do want them to have external linkage you can always declare them in the class specifier as normal.

Your proposal breaks a current rule that a qualified declarator id has to be declared unqualified previously in the TU.  This might cause a minor hassle to implementors with name lookup on declarator ids during parsing.

I think the main resistance you will receive is from people that claim that this would break encapsulation.  Currently the only entities that can access private members are all declared in the class specifier (either as members or friends).  Some will claim that this is a nice property for programming in the large, as you can isolate what can be messing with private data to a list of possible culprits all in one place.  A private extension method, on the other hand, can be declared anywhere in the codebase.  I'm however on your side fwiw, and think that having private member functions not cluttering the class specifier outweighs the loss.

Also you might want to spend some time studying the interaction with template classes.  I assume you can also define private extension methods of template classes too?  How will they interact with specialization/instantiation?

Good luck,

Andrew.

[fmatth...@gmail.com]

On Monday, December 9, 2013 10:26:26 PM UTC-5, Andrew Tomazos wrote:

Hi Matthew,

I did briefly read your proposal.

You touch on this a little in your proposal, but just to be clear 9.4p5 [class.static]:

> When used in the declaration of a class member, the static specifier shall only be used in the member declarations that appear within the member-specification of the class definition.

So this is currently ill-formed:

struct X

{

    void f();

};

static void X::f() {} // ERROR

This frees the static specifier up to be used to specify a static member function if that is what you want.  There is no ambiguity with the internal linkage meaning on a free function.  In the event someone accidentally places static in front of a private extension method, intending internal linkage but getting a static member function, then the only effect is a compile-time error when they try to access the implicit object parameter that doesn't exist.  It isn't dangerous.

True, but then we are unable to use the static keyword to specify internal linkage for private extension methods. We will be forced to use an anonymous namespace. I usually prefer the static keyword for file scope functions to anonymous namespaces as it avoids an extra scope, but that's minor style point. Making the static prefix indicate a static method is the path of least resistance and I'd be happy go that way should the committee prefer that direction.

You might also want to consider giving private extension methods implicit internal linkage (both static and non-static member functions), the only use case I can imagine is placing them in the .cpp so the linker doesn't even need to see the symbol, and if you really do want them to have external linkage you can always declare them in the class specifier as normal.

Not sure I agree here. I have had plenty of situations where my class is implemented in multiple .cc files. In these situations, you may want to declare one or more private extension methods in an internal header use them in multiple translation units. You can also

declare extension methods directly in the public header file if you want to access them from inlined public/protected methods.

 

Your proposal breaks a current rule that a qualified declarator id has to be declared unqualified previously in the TU.  This might cause a minor hassle to implementors with name lookup on declarator ids during parsing.

If necessary, I'd be happy to consider writing a reference implementation patchfor clang and/or gcc to show feasibility. I'm no compiler writer but it seems to me that this should not be a difficult feature to implement.

 

 

I think the main resistance you will receive is from people that claim that this would break encapsulation.  

Indeed, I've taken great pains to show that this feature does in fact not break encapsulation in order to counter that

argument. In fact I argue it even strengthens encapsulation because it removes unnecessary details from the class interface.

 

Currently the only entities that can access private members are all declared in the class specifier (either as members or friends).  Some will claim that this is a nice property for programming in the large, as you can isolate what can be messing with private data to a list of possible culprits all in one place.  

I find this to be more restrictive then helpful. Procedural C APIs can be implemented with multiple functions under the hood, why not allow the same for C++ object oriented APIs? It seems like we just have this arbitrary rule that restricts the number of functions we are allow to use to implement our class behavior.

 

A private extension method, on the other hand, can be declared anywhere in the codebase.  I'm however on your side fwiw, and think that having private member functions not cluttering the class specifier outweighs the loss.

Also you might want to spend some time studying the interaction with template classes.  I assume you can also define private extension methods of template classes too?  How will they interact with specialization/instantiation?

Yes templates will be supported. One should be able to specialize using a syntax like the following:

template <typename T>

class Foo {
};

//A private extension method only for specialized Foo<int>

template <>
Foo<int>::_f() {
}

Thanks for mentioning this. Its a good point which needs to be analyzed and addressed. I will add a section for it.

 

Good luck,

Andrew.

Thank you for your feedback, much appreciated!

[Magnus Fromreide]

On Mon, 2013-12-09 at 18:20 -0800, fmatth...@gmail.com wrote:
> Hello everyone, I am going to submit a proposal for relaxing the
> restriction on requiring private non-virtual class methods and static
> private class methods to be declared in the class definition.
>
> The github repository with the current draft of the proposal is here:
> https://github.com/fmatthew5876/stdcxx-privext

How does this interact with overload resolution?
Consider the following:

---vvv--- foo.hh ---vvv---
class foo {
public:
void bar() { gaz(17); }
void gaz(long);
private:
some_type secret;
};

foo getAFoo();
---^^^--- foo.hh ---^^^---
and
---vvv--- evil.cc ---vvv---
#include "foo.hh"

void doTheDeed(some_type&);

void foo::gaz(int i)
{
doTheDeed(secret);
}

void doEvil() { getAFoo().bar(); }
---^^^--- evil.cc ---^^^---

Does a call to doEvil() result in a call to gaz(long) or gaz(int)?



I would also like to ask if I understand you correctly in that you are
suggesting to allow

#include "foo.hh"
namespace {
void foo::gazonk() { /* something */ }
}

and here foo::gazonk is a private member of foo?

/MF

[fmatth...@gmail.com]

It should call gaz(long), because that is what was visible at the time foo() was inlined.

Its the same as this:

void gaz(long);

void foo() { gaz(17); {

void gaz(int);

I'll add a section talking about overloading.

I would also like to ask if I understand you correctly in that you are
suggesting to allow

#include "foo.hh"
namespace {
  void foo::gazonk() { /* something */ }
}

and here foo::gazonk is a private member of foo?

That's right, although it does seem kind of wierd because any foo member function should be in the same namespace as foo. This would require a special exception for anonymous name spaces. Internal linkage is a very important feature I want to be sure to enable. In order to do that we need either the static keyword or anonymous namespace support.

/MF

 

[corn...@google.com]

On Tuesday, December 10, 2013 3:20:46 AM UTC+1, fmatth...@gmail.com wrote:

Hello everyone, I am going to submit a proposal for relaxing the restriction on requiring private non-virtual class methods and static private class methods to be declared in the class definition.

The github repository with the current draft of the proposal is here:

https://github.com/fmatthew5876/stdcxx-privext

Hi,

I love this proposal and wanted to do pretty much the same thing for some time.

In "Alternate Syntax", you say: "One downside of the current approach is that previously if the user wrote a typo when writing a member function name in the definition they would get a helpful compilation error."

In my opinion, this is a deal breaker. I think it's absolutely necessary to use a keyword for this, and I strongly recommend using 'private'. I consider the objection that this will confuse people to be weak; compilers can easily be taught to acknowledge the syntax with more than an "unexpected 'public'" and say something like "only private member functions can be declared outside the class". If a user is still curious as to why that is so, StackOverflow or std-discussion will readily supply the necessary "Because it breaks encapsulation" answer.

On a side note, if you want some evidence for the value of this proposal, you should check out the Sema implementation of Clang. There's lots of static helper functions in the implementation files, they all get passed the Sema object, and quite a few of them should have been private members, because they force Sema members to be public that shouldn't have been.

[Klaim - Joël Lamotte]

This proposal is interesting.

Minor side comment: I'm not sure if the word "method" is used correctly here. I see what you mean but my understanding is that depending on the language, method can mean virtual member function.
I remember C++ people saying that in C++ we should use member function to clarify that it have nothing to do with dynamic dispatch.

More importantly:

I would like to see a paragraph exploring if it feature could or not allow of the interface "leak", potentially breaking the encapsulation, maybe not in a harmful way, I'm not sure.

I mean: at first read of this proposal, it looks like it's not possible to call an extended private member function without it being called by another member function that ultimately
would be in the class interface.

However, it seems possible to break the encapsulation this way:

///////////////////////////////////////////////////////////////////

// Header "mylib.hpp" from an library "MyLib"

namespace mylib

{

   class Foo

   {

       // ...

   private:

        int k; // never exposed in the public interface

   };

}

///////////////////////////////////////////////////////////////////////

// header "mycode.hpp" in the user code

#include <mylib.hpp>

namespace mylib

{

    int* Foo::get_k() { return &this->k; } // private extension

}

///////////////////////////////////////////////////////////////////

// cpp in the user code

#include <mycode.hpp>

   void break_in( mylib::Foo& foo )

   {

        auto private_k_ptr =  foo.get_k();  // accessing private members "legally"

   }

////////////////////////////////////////////////////////////////////////

Could you clarify what would prevent this to be allowed? If it is allowed, then maybe clarifying if it's not a problem?
And if it is a problem, then I think the proposal is problematic.
However I'm not expert so I certainly missed something?

[Klaim - Joël Lamotte]

On Tue, Dec 10, 2013 at 10:53 PM, Klaim - Joël Lamotte <mjk...@gmail.com> wrote:

I would like to see a paragraph exploring if it feature could or not allow of the interface "leak", potentially breaking the encapsulation, maybe not in a harmful way, I'm not sure.

Sorry, I forgot to fix this sentence, I meant:

I would like to see a paragraph exploring how the feature could or not allow of the interface to "leak" it's data, potentially breaking the encapsulation, maybe not in a harmful way, I'm not sure.


To add to my previous comment:

Basically it looks to me that until we have a way to hide the types of the private member data, I can't see how it would be possible to avoid this kind of encapsulation breakage.

[Nevin Liber]

On 9 December 2013 21:26, Andrew Tomazos <andrew...@gmail.com> wrote:

I think the main resistance you will receive is from people that claim that this would break encapsulation.  

Claim?  Are you claiming this doesn't break encapsulation?

--
 Nevin ":-)" Liber  <mailto:ne...@eviloverlord.com>  (847) 691-1404

[Vicente J. Botet Escriba]

Le 10/12/13 03:20, fmatth...@gmail.com a écrit :

Hi,

there is a C++ idiom that provides most of the advantages of your proposal (see http://viboes.blogspot.fr/2010/04/pimpl-variant.html).
It consists in declaring a friend detail class.

//foo.hh
class Foo {
  struct detail;
  friend detail;

  public:
    Foo();
    void pub1();
    void pub2();
  private:

    int _i;
    int _j;
};

//foo.cc

struct Foo::detail {

  Foo& that;

  detail(Foo* f) : that(*f) {}

  //Definition of _priv1()
  void _priv1() {
    /* function body */
  }

  //A new scoped private extension method.
  static void _priv3() {
    that._i = 3;
  }

};


//Definition of Foo::pub1()
void Foo::pub1() {
  detail(this)._priv1();
  detail(this)._priv3();
}

This idiom solves most of the problem the proposal address and doesn't have the drawback pointed by Jöel, as the detail class is private and no other can add private functions.

There are some holes on this idiom as friend is not transitive so other friend classes could not access the backdoor functions neither.

I don't see any benefit to inline calls to private functions that would be defined in a .cc file as your

//Forward declaration of a private extension method
void Foo::_priv2();

//Inlined public method calls the private extension method
inline void Foo::pub2() { _priv2(); }

inlining the _priv2() call would result on the same code.

I don't see neither the needed to declare new private functions on different files. Could you explain where this could be useful.

Maybe the idiom could be adapted to a language proposal that could avoid the cumbersome use of the that variable. It could consists in declaring a pseudo-struct declaration e.g. private Foo, that could contain only functions. These functions would be accessible only by the class Foo and any friend of Foo. Of course, in order to other classes to be able to use these private functions, the private pseudo-struct should be declared in another file.

//foo.hh
class Foo {
  public:
    Foo();
    void pub1();
    void pub2();
  private:

    int _i;
    int _j;
};

//foo.cc

private Foo {

  //Definition of _priv1()
  void _priv1() {
    /* function body */
  }

  //A new scoped private extension method.
  static void _priv3() {
    _i = 3;
  }

};


//Definition of Foo::pub1()
void Foo::pub1() {
  _priv1();
  _priv3();
}

It is clear that all this would have less sense with modules.

Best,
Vicente

[Thiago Macieira]

On terça-feira, 10 de dezembro de 2013 22:53:34, Klaim - Joël Lamotte wrote:
> Could you clarify what would prevent this to be allowed? If it is allowed,
> then maybe clarifying if it's not a problem?

Looks like it would be allowed and it should not be considered a problem. We
give the guns, if people point to their feet and pull the trigger, we can't be
blamed.

That can already be done if the interface designer left a loophole: a class
friend that isn't defined in this compilation unit.

class Bar;

class Foo
{
// ...
private:
int k;

friend Bar;
};


/// hack.cpp
class Bar
{
static int &get_k(Foo *d) { return d->k; }
};
int &get_foo_k(Foo *d)
{
return Bar::get_k(d);
}

To be honest, I don't know what the standard says about two different
declarations of the same class in the same application but not the same
translation unit. But the above will compile in most ABIs.

Not to mention #define private public
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[Vicente J. Botet Escriba]

Le 10/12/13 23:46, Thiago Macieira a �crit :

> On ter�a-feira, 10 de dezembro de 2013 22:53:34, Klaim - Jo�l Lamotte wrote:
>> Could you clarify what would prevent this to be allowed? If it is allowed,
>> then maybe clarifying if it's not a problem?
> Looks like it would be allowed and it should not be considered a problem. We
> give the guns, if people point to their feet and pull the trigger, we can't be
> blamed.

I disagree. The goal is not to give any gun. Just to be able to refactor
the implementation without showing it at the interface level.

> That can already be done if the interface designer left a loophole: a class
> friend that isn't defined in this compilation unit.
>
> class Bar;
> class Foo
> {
> // ...
> private:
> int k;
> friend Bar;
> };
>
>
> /// hack.cpp
> class Bar
> {
> static int &get_k(Foo *d) { return d->k; }
> };
>
>

> To be honest, I don't know what the standard says about two different
> declarations of the same class in the same application but not the same
> translation unit. But the above will compile in most ABIs.

This violated the ODR.

> Not to mention #define private public

Note that the original idea is to don't show the private functions, not
making all private function public.

Vicente

[Nevin Liber]

On 10 December 2013 16:46, Thiago Macieira <thi...@macieira.org> wrote:

give the guns, if people point to their feet and pull the trigger, we can't be
blamed.

That can already be done if the interface designer left a loophole: a class
friend that isn't defined in this compilation unit.

class Bar;

class Foo
{
    // ...
private:
    int k;

    friend Bar;
};

It isn't a loophole.  Foo has explicitly given permission for Bar to access the innards of Foo.  That just isn't true for the proposal.

Even though there are some encapsulation loopholes in the language (member templates come to mind), as Herb Sutter puts it in <http://www.gotw.ca/gotw/076.htm>: "The issue here is of protecting against Murphy vs. protecting against Machiavelli... that is, protecting against accidental misuse (which the language does very well) vs. protecting against deliberate abuse (which is effectively impossible)."

I am strongly against making it this trivial to ignore encapsulation.

[Klaim - Joël Lamotte]

On Tue, Dec 10, 2013 at 11:46 PM, Thiago Macieira <thi...@macieira.org> wrote:

That can already be done if the interface designer left a loophole: a class
friend that isn't defined in this compilation unit.

Yes, but the most common case I've seen in real use was more exploiting the visibility rules of private inner classes:

// header

class Foo

{

    // ...

private:

    int k;

    class Bar; // inner class, private so even if you have Bar interface, you can't use it from outside Foo's implementation

};

// cpp

class Foo::Bar

{

   // takes a Foo instance and access it's private members.

};

// user cpp

class Foo::Bar // ODR violation, maybe hidden in case of dynamic libraries?

{

 // do whatever with Foo

 // can't use it's interface as Foo::Bar name is not accessible anyway, so only Foo members

  // can use this type

 // If it's a dynamic library, this implementation will never be used by non-inline Foo member functions.

};

Here Foo::Bar is both acting like a friend of Foo and can't be used by external code as

even if you define Foo::Bar, the name "Foo::Bar" can't be used outside member functions of Foo.

Assuming you define Foo::Bar with an interface that match Foo's members use of Foo::Bar, but only

the ones defined in the  header (and potentially inlined), you'd have a hard time making this work
correctly in conjonction with the original Foo::Bar.
Basically, it's not really easy to get into Foo through Foo::Bar.

However, it is really easy to get any private data exposed in the class definition using extends methods,

without having to change the class definition at all (which is worse than friend which at least force the user

to change the library code, which have huge impact on maintenance, which is a big pressure not to do so

if it's not for a bugfix but for abuse).

I'm not sure if it would be possible to modify your proposal so that it gives similar garantees to inner classes, 

but with easier syntax? At the moment I can't think of a good way.

Side comment: It looks like the issue the proposal is trying to partially fix is really hard to improve without changing at least a bit 
the compilation model itself.

[Klaim - Joël Lamotte]

Let me clarify my example a bit:

On Wed, Dec 11, 2013 at 12:27 AM, Klaim - Joël Lamotte <mjk...@gmail.com> wrote:

// header

class Foo

{

    // ...

        void action(); 

private:

    int k;

    class Bar; // inner class, private so even if you have Bar interface, you can't use it from outside Foo's implementation

};

// cpp

class Foo::Bar

{

   // takes a Foo instance and access it's private members.

       Foo& m_foo; 

       Bar( Foo& foo ) : m_foo( foo ) {} 

       Bar& inc_counter() { ++m_foo.k; return this; }

   

};

    

    void Foo::action() { Bar(*this).inc_counter(); } // here we use Foo::Bar into Foo, so no compilation error

     

// user cpp

class Foo::Bar // ODR violation, maybe hidden in case of dynamic libraries?

{

 // do whatever with Foo

 // can't use it's interface as Foo::Bar name is not accessible anyway, so only Foo members

  // can use this type

 // If it's a dynamic library, this implementation will never be used by non-inline Foo member functions.

};

If nobody seen any problem with this way of doing, maybe a syntax to simplify this would help.
Something that would look like:

// header

class Foo

{

    // ...

        void action(); 

private:

    int k;

    extension Bar; // like an inner class, private so even if you have Bar interface, you can't use it from outside Foo's implementation

    // extension can't be used in non-private section.

};

// cpp

extension Foo::Bar

   // like an inner class BUT:

   // - can only contain private members functions and private static functions for Foo

   // - don't allow visibility scope (as everything is private)

   // - can't be inherited? etc.

   // - members of Foo::Bar are implicitely usable from Foo members defined AFTER this definition

   // - this is Foo's instance calling the private member or static function (see the usage)

{

      

       // No need for the following: Bar code is implicitely usable from

       // Foo& m_foo; 

       // Bar( Foo& foo ) : m_foo( foo ) {}   

       Bar& inc_counter() { ++m_foo.k; return this; }

   

};

    

    void Foo::action() { inc_counter(); } // here we use Foo::Bar, but in an implicit way

     

// user cpp

extension Foo::Bar // ODR violation, maybe hidden in case of dynamic libraries?

{

 // do whatever with Foo

 // can't use it's interface as Foo::Bar name is not accessible anyway, so only Foo members

  // can use this type

 // If it's a dynamic library, this implementation will never be used by non-inline Foo member functions.
};

It's just a shortcut of using inner class, so not sure it's worth (it does help to have implicit this to the outter class instance though...)

So it's basically a shortcut of an idiom I've seen and used, but I didn't explore the potential problems (other than adding a keyword).

What do you think?

[inkwizyt...@gmail.com]

isnt private extension still private? I every one can create private function but only private function can call it you can only create dead code.
To even call any new private extension you need have at least one normal member function to call it.

[Klaim - Joël Lamotte]

On Wed, Dec 11, 2013 at 12:47 AM, <inkwizyt...@gmail.com> wrote:

isnt private extension still private? I every one can create private function but only private function can call it you can only create dead code.
To even call any new private extension you need have at least one normal member function to call it.

I think you are correct, my first example would have an error on the foo.get_k() call.

So basically it works in the similar way than my last examples with inner classes (without the verbosity).

I initially thought that using member function pointer could be used to hack an access to the private data, 

but I'm not sure.

I should just sleep more.

[Ville Voutilainen]

On 11 December 2013 00:59, Vicente J. Botet Escriba
<vicent...@wanadoo.fr> wrote:
> Le 10/12/13 23:46, Thiago Macieira a écrit :

>> class Bar;
>> class Foo
>> {
>> // ...
>> private:
>> int k;
>> friend Bar;
>> };
>>
>>
>> /// hack.cpp
>> class Bar
>> {
>> static int &get_k(Foo *d) { return d->k; }
>> };
>>
>>
>> To be honest, I don't know what the standard says about two different
>> declarations of the same class in the same application but not the same
>> translation unit. But the above will compile in most ABIs.
>
> This violated the ODR.

Which bit of it does that, exactly?

[Richard Smith]

On Tue, Dec 10, 2013 at 3:47 PM, <inkwizyt...@gmail.com> wrote:

isnt private extension still private? I every one can create private function but only private function can call it you can only create dead code.
To even call any new private extension you need have at least one normal member function to call it.

This isn't true; you can use this extension to violate encapsulation (as I demonstrated on the previous discussion thread on this topic). That said, it's difficult, and it's *very* unlikely that you'd do it by accident.

Here's one way to violate encapsulation with this extension:

class A {

  int n;

public:

  A() : n(42) {}

};

template<typename T> struct X {

  static decltype(T()()) t;

};

template<typename T> decltype(T()()) X<T>::t = T()();

int A::*p;

private int A::expose_private_member() { // note, not called anywhere

  struct DoIt {

    int operator()() {

      p = &A::n;

      return 0;

    }

  };

  return X<DoIt>::t; // odr-use of X<DoIt>::t triggers instantiation

}

int main() {

  A a;

  return a.*p; // read private member

}

 

--
 
---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.

[David Krauss]

The example is unclear, but it seems to imply Bar must also be defined
somewhere besides the hack. A class (named) with external linkage shall
have an identical definition in every TU. (3.2/6)

A better way to violate privileges without violating the ODR would be to
explicitly specialize a member of Bar which cannot be named by another
TU. There are other tricks based on specialization.

What if a derived class meeting some strict requirements could be
defined as alias-compatible with the base, such that an object of a base
class could be used as the interface, and suitably casted used as the
implementation?

Extension methods are nice to have and closely related to modules.
Perhaps someone can draft a modules-compatible, self-contained extension
providing private implementation semantics, and get a head start on ABI
work related to consequences in the type system.

[fmatth...@gmail.com]

Wow lots of comments. 

Before I answer the comments, I'd like to emphasize this article which someone else has already mentioned. http://www.gotw.ca/gotw/076.htm

Here, Herb Sutter talks about ways in current C++ that you can break encapsulation. I don't believe my feature adds any new ways to get at private data that don't already exist.

What we want to avoid is allowing an easy to use "break encapsulation here" feature. If someone can break encapsulation by doing the horrible things mentioned in that article, that's ok. Access control in C++ is not a security feature, its a productivity and organization feature. It helps you write good code, its not designed to save lives.

With that, lets get started..

On Tuesday, December 10, 2013 10:17:29 AM UTC-5, corn...@google.com wrote:

Hi,

I love this proposal and wanted to do pretty much the same thing for some time.

Good to know someone is annoyed by this restriction as I am.

 

In "Alternate Syntax", you say: "One downside of the current approach is that previously if the user wrote a typo when writing a member function name in the definition they would get a helpful compilation error."

In my opinion, this is a deal breaker.

I don't think its a huge deal breaker, since the error will be caught at link time. But I'm not too concerned with the syntax, as long as this feature makes it in.

 

I think it's absolutely necessary to use a keyword for this, and I strongly recommend using 'private'.

Actually using a keyword nicely solves the static dilemna:

private Foo::f1() //private extension method

private static Foo::f2() //static private method

static private Foo::f3() //private extension method with internal linkage

static private static Foo::f4() //static private extension method with internal linkage

Going to think about it more, maybe using the private keyword makes sense.

 

On a side note, if you want some evidence for the value of this proposal, you should check out the Sema implementation of Clang. There's lots of static helper functions in the implementation files, they all get passed the Sema object, and quite a few of them should have been private members, because they force Sema members to be public that shouldn't have been.

Interesting, I'll take a look at this. A real world example would add a lot of support to the proposal. Thank you for bringing it to my attention.

 On Tuesday, December 10, 2013 4:53:34 PM UTC-5, Klaim - Joël Lamotte wrote:

This proposal is interesting.

Minor side comment: I'm not sure if the word "method" is used correctly here. I see what you mean but my understanding is that depending on the language, method can mean virtual member function.
I remember C++ people saying that in C++ we should use member function to clarify that it have nothing to do with dynamic dispatch.

I haven't heard that before but you might be right. I'll double check and fix the wording. Thanks.

 

More importantly:

I would like to see a paragraph exploring if it feature could or not allow of the interface "leak", potentially breaking the encapsulation, maybe not in a harmful way, I'm not sure.

I mean: at first read of this proposal, it looks like it's not possible to call an extended private member function without it being called by another member function that ultimately
would be in the class interface.

The extension methods are private and thus can only be access by members of the class. I don't see how encapsulation could be broken by being able to add more private methods. I'd love it if someone could come up with a counter example. I haven't been able to find any.

 

However, it seems possible to break the encapsulation this way:

///////////////////////////////////////////////////////////////////

// Header "mylib.hpp" from an library "MyLib"

namespace mylib

{

   class Foo

   {

       // ...

   private:

        int k; // never exposed in the public interface

   };

}

///////////////////////////////////////////////////////////////////////

// header "mycode.hpp" in the user code

#include <mylib.hpp>

namespace mylib

{

    int* Foo::get_k() { return &this->k; } // private extension

}

///////////////////////////////////////////////////////////////////

// cpp in the user code

#include <mycode.hpp>

   void break_in( mylib::Foo& foo )

   {

        auto private_k_ptr =  foo.get_k();  // accessing private members "legally"

   }

////////////////////////////////////////////////////////////////////////

Could you clarify what would prevent this to be allowed? If it is allowed, then maybe clarifying if it's not a problem?

And if it is a problem, then I think the proposal is problematic.
However I'm not expert so I certainly missed something?

This will not compile. In break_in() you're calling a Foo::get_k() which is private. The fact that it is defined outside of the class doesn't change anything.

 

On Tuesday, December 10, 2013 4:57:30 PM UTC-5, Klaim - Joël Lamotte wrote:

To add to my previous comment:

Basically it looks to me that until we have a way to hide the types of the private member data, I can't see how it would be possible to avoid this kind of encapsulation breakage.

If you want to hide the types of private data you have to use PIMPL. This is already supported.

On Tuesday, December 10, 2013 5:11:44 PM UTC-5, Nevin ":-)" Liber wrote:

On 9 December 2013 21:26, Andrew Tomazos <andrew...@gmail.com> wrote:

I think the main resistance you will receive is from people that claim that this would break encapsulation.  

Claim?  Are you claiming this doesn't break encapsulation?

I am claiming with all seriousness that this doesn't break encapsulation. The number of private methods and their signatures are implementation details because they are not accessible by direct users or child classes who inherit. In addition, non-virtual private methods are not required to be in the class definition by the compiler in the same way that the private data members are. Implementation details do not belong in the interface. For that reason, my proposal actually provides better encapsulation than the current C++ class model.

 

I've seen these kinds of idioms before. They go most of the way but there are still some drawbacks:

* The syntax is not natural. Instead of direct access to the data members you have to indirect through your 'that' pointer.

* You still have to declare the friend class in your class definition. Thus there is still an implementation dependency required in the interface. If you want to make any changes, it changes the interface and forces a recompile to the users.

* The set of extensions are now fixed within your friend class definition and as you mentioned before, friends are not transitive so you cannot make another friend to extend further. Private extension methods give you the ultimate freedom in implementation without breaking encapsulation. They don't depend on hacks like friends and nested classes.

That being said, I should probably list some of these examples and explain why my proposal is better.

I don't see any benefit to inline calls to private functions that would be defined in a .cc file as your

//Forward declaration of a private extension method
void Foo::_priv2();

//Inlined public method calls the private extension method
inline void Foo::pub2() { _priv2(); }

inlining the _priv2() call would result on the same code.

I don't see neither the needed to declare new private functions on different files. Could you explain where this could be useful.

I've had plenty of situations where a class is implemented in multiple cc files in order to better separate concerns. I find myself doing this more for classes that implement "Application Logic." Usually singular concepts such as data structures are implemented in one cc file, if not entirely in the header due to templates.

In this situation, you would have your public header with the class definition, a private header with some extension method declarations, and finally multiple cc files with the definitions and possibly more file scoped extension methods as needed.

Declaring an extension method in the same header as the class definition has little if no value. That's merely a stylistic choice. I included it just to show the possibilities.

 

Maybe the idiom could be adapted to a language proposal that could avoid the cumbersome use of the that variable. It could consists in declaring a pseudo-struct declaration e.g. private Foo, that could contain only functions. These functions would be accessible only by the class Foo and any friend of Foo. Of course, in order to other classes to be able to use these private functions, the private pseudo-struct should be declared in another file.

//foo.hh
class Foo {
  public:
    Foo();
    void pub1();
    void pub2();
  private:

    int _i;
    int _j;
};

//foo.cc

private Foo {

  //Definition of _priv1()
  void _priv1() {
    /* function body */
  }

  //A new scoped private extension method.
  static void _priv3() {
    _i = 3;
  }

};

 

This is an interesting idea. Essentially reopening the scope to add more private stuff. Not only could you add new private methods but also private typedefs and nested classes as well. In the old thread (see github for the link), someone was essentially proposing this idea. He wanted to use a concept of class namespaces, but I like your syntax better.

//Definition of Foo::pub1()
void Foo::pub1() {
  _priv1();
  _priv3();
}

It is clear that all this would have less sense with modules.

Modules. Who knows what they are or when we will have them. I want something to solve my problems now. Also this can be a step towards modules.

 

On Tuesday, December 10, 2013 6:15:34 PM UTC-5, Nevin ":-)" Liber wrote:

I am strongly against making it this trivial to ignore encapsulation.

Please provide an example where a private extension method can be used to easily ignore encapsulation, outside of the already existing exploits mentioned in Herb's article.

On Tuesday, December 10, 2013 6:27:49 PM UTC-5, Klaim - Joël Lamotte wrote:

However, it is really easy to get any private data exposed in the class definition using extends methods,

without having to change the class definition at all (which is worse than friend which at least force the user

to change the library code, which have huge impact on maintenance, which is a big pressure not to do so

if it's not for a bugfix but for abuse).

Again, do you have an example using my proposal that makes it easy to violate access control? I would like to see an example that doesn't involve violating the ODR rule, defining private to public, or using template specialization as in Herbs Article.

 

I'm not sure if it would be possible to modify your proposal so that it gives similar garantees to inner classes, 

but with easier syntax? At the moment I can't think of a good way.

I don't think inner classes are affected by this proposal.

 

Side comment: It looks like the issue the proposal is trying to partially fix is really hard to improve without changing at least a bit 
the compilation model itself.

Not sure what you mean, can you clarify?

 On Tuesday, December 10, 2013 7:20:23 PM UTC-5, Ville Voutilainen wrote:

On 11 December 2013 00:59, Vicente J. Botet Escriba 
<vicent...@wanadoo.fr> wrote: 
> Le 10/12/13 23:46, Thiago Macieira a écrit : 
>> class Bar; 
>> class Foo 
>> { 
>>      // ... 
>> private: 
>>      int k; 
>>      friend Bar; 
>> }; 
>> 
>> 
>> /// hack.cpp 
>> class Bar 
>> { 
>>      static int &get_k(Foo *d) { return d->k; } 
>> }; 
>> 
>> 
>> To be honest, I don't know what the standard says about two different 
>> declarations of the same class in the same application but not the same 
>> translation unit. But the above will compile in most ABIs. 
> 
> This violated the ODR. 



Which bit of it does that, exactly? 

If Foo declares Bar as a friend. Its likely that Bar is also going to be defined in the same library or application. You're redefining Bar in the hack.cc file and thus you violated the ODR. 

On Tuesday, December 10, 2013 7:33:31 PM UTC-5, Richard Smith wrote:

Here's one way to violate encapsulation with this extension:

class A {

  int n;

public:

  A() : n(42) {}

};

template<typename T> struct X {

  static decltype(T()()) t;

};

template<typename T> decltype(T()()) X<T>::t = T()();

int A::*p;

private int A::expose_private_member() { // note, not called anywhere

  struct DoIt {

    int operator()() {

      p = &A::n;

      return 0;

    }

  };

  return X<DoIt>::t; // odr-use of X<DoIt>::t triggers instantiation

}

int main() {

  A a;

  return a.*p; // read private member

}

 

There is our example. I tested it and it does work (you can test it by just declaring expose_private_member() in the class definition). Still this is so complex that I think it belongs as the 4th example Herb's article.

I will include this example in the proposal.

On Tuesday, December 10, 2013 7:34:33 PM UTC-5, David Krauss wrote:

A better way to violate privileges without violating the ODR would be to 
explicitly specialize a member of Bar which cannot be named by another 
TU. There are other tricks based on specialization. 

This is also mentioned in Herb's article. Nothing new from my proposal.

 

What if a derived class meeting some strict requirements could be 
defined as alias-compatible with the base, such that an object of a base 
class could be used as the interface, and suitably casted used as the 
implementation? 

You can do that now if you change your private members to protected and then inherit. This exposes your private members to legatimate child classes though and is a huge break in encapsulation.

 

Extension methods are nice to have and closely related to modules. 
Perhaps someone can draft a modules-compatible, self-contained extension 
providing private implementation semantics, and get a head start on ABI 
work related to consequences in the type system. 

I'm going ahead with this proposal. I don't think it gets in the way of modules.

Thank you everyone for your feedback some great ideas here. I will try to incorporate all of the feedback into the next draft. 

 

[Vicente J. Botet Escriba]

Le 11/12/13 01:20, Ville Voutilainen a �crit :

> On 11 December 2013 00:59, Vicente J. Botet Escriba
> <vicent...@wanadoo.fr> wrote:

>> Le 10/12/13 23:46, Thiago Macieira a �crit :

>>> class Bar;
>>> class Foo
>>> {
>>> // ...
>>> private:
>>> int k;
>>> friend Bar;
>>> };
>>>
>>>
>>> /// hack.cpp
>>> class Bar
>>> {
>>> static int &get_k(Foo *d) { return d->k; }
>>> };
>>>
>>>
>>> To be honest, I don't know what the standard says about two different
>>> declarations of the same class in the same application but not the same
>>> translation unit. But the above will compile in most ABIs.
>> This violated the ODR.
>
>
> Which bit of it does that, exactly?
>

I expect that the class Bar will be defined somewhere else and that it
is not only here to hack, but maybe I miss something.

Vicente

[Vicente J. Botet Escriba]

Le 11/12/13 03:55, fmatth...@gmail.com a écrit :

Wow lots of comments. 

I don't see any benefit to inline calls to private functions that would be defined in a .cc file as your

//Forward declaration of a private extension method
void Foo::_priv2();

//Inlined public method calls the private extension method
inline void Foo::pub2() { _priv2(); }

inlining the _priv2() call would result on the same code.

I don't see neither the needed to declare new private functions on different files. Could you explain where this could be useful.

I've had plenty of situations where a class is implemented in multiple cc files in order to better separate concerns. I find myself doing this more for classes that implement "Application Logic." Usually singular concepts such as data structures are implemented in one cc file, if not entirely in the header due to templates.

In this situation, you would have your public header with the class definition, a private header with some extension method declarations, and finally multiple cc files with the definitions and possibly more file scoped extension methods as needed.

I understand your use case. Reducing compilation dependencies between implementation of different classes seems to me the problem to address. While taking care of the case where multiple files implements a specific class would be nice, it is for me a corner case. 

Declaring an extension method in the same header as the class definition has little if no value. That's merely a stylistic choice. I included it just to show the possibilities.

So we agree this is not a must have.

 

Maybe the idiom could be adapted to a language proposal that could avoid the cumbersome use of the that variable. It could consists in declaring a pseudo-struct declaration e.g. private Foo, that could contain only functions. These functions would be accessible only by the class Foo and any friend of Foo. Of course, in order to other classes to be able to use these private functions, the private pseudo-struct should be declared in another file.

//foo.hh
class Foo {
  public:
    Foo();
    void pub1();
    void pub2();
  private:

    int _i;
    int _j;
};

//foo.cc

private Foo {

  //Definition of _priv1()
  void _priv1() {
    /* function body */
  }

  //A new scoped private extension method.
  static void _priv3() {
    _i = 3;
  }

};

 

This is an interesting idea. Essentially reopening the scope to add more private stuff. Not only could you add new private methods but also private typedefs and nested classes as well. In the old thread (see github for the link), someone was essentially proposing this idea. He wanted to use a concept of class namespaces, but I like your syntax better.

Note that for me the scope could be reopened only once. In order to avoid that the user can do it on any class it would be better to restrict it to class that have declared it. E.g. we could declare the class friend of itself.

//foo.hh
class Foo {
	friend Foo;

};

//Definition of Foo::pub1()
void Foo::pub1() {
  _priv1();
  _priv3();
}

It is clear that all this would have less sense with modules.

Modules. Who knows what they are or when we will have them. I want something to solve my problems now. Also this can be a step towards modules.

You are right, but you will not have a new feature before 2017 as minor as it can be.

Vicente

[fmatth...@gmail.com]

On Wednesday, December 11, 2013 1:19:14 AM UTC-5, Vicente J. Botet Escriba wrote:

Le 11/12/13 03:55, fmatth...@gmail.com a écrit :

Wow lots of comments. 

I don't see any benefit to inline calls to private functions that would be defined in a .cc file as your

//Forward declaration of a private extension method
void Foo::_priv2();

//Inlined public method calls the private extension method
inline void Foo::pub2() { _priv2(); }

inlining the _priv2() call would result on the same code.

I don't see neither the needed to declare new private functions on different files. Could you explain where this could be useful.

I've had plenty of situations where a class is implemented in multiple cc files in order to better separate concerns. I find myself doing this more for classes that implement "Application Logic." Usually singular concepts such as data structures are implemented in one cc file, if not entirely in the header due to templates.

In this situation, you would have your public header with the class definition, a private header with some extension method declarations, and finally multiple cc files with the definitions and possibly more file scoped extension methods as needed.

I understand your use case. Reducing compilation dependencies between implementation of different classes seems to me the problem to address. While taking care of the case where multiple files implements a specific class would be nice, it is for me a corner case. 

Reducing dependencies by minimizing the interface is the name of the game here. I agree that multiple source files is a corner case, but I'd like to support it as it is useful sometimes. 

Declaring an extension method in the same header as the class definition has little if no value. That's merely a stylistic choice. I included it just to show the possibilities.

So we agree this is not a must have.

Yes, but if you can declare a private extension method just like a regular free function, there's no reason shouldn't be able to do it in the class header. I suppose one corner use case is conditional compilation. One version may want to call an inline PEM, while another does not (PEM is #ifdefed out).

 

Maybe the idiom could be adapted to a language proposal that could avoid the cumbersome use of the that variable. It could consists in declaring a pseudo-struct declaration e.g. private Foo, that could contain only functions. These functions would be accessible only by the class Foo and any friend of Foo. Of course, in order to other classes to be able to use these private functions, the private pseudo-struct should be declared in another file.

//foo.hh
class Foo {
  public:
    Foo();
    void pub1();
    void pub2();
  private:

    int _i;
    int _j;
};

//foo.cc

private Foo {

  //Definition of _priv1()
  void _priv1() {
    /* function body */
  }

  //A new scoped private extension method.
  static void _priv3() {
    _i = 3;
  }

};

 

This is an interesting idea. Essentially reopening the scope to add more private stuff. Not only could you add new private methods but also private typedefs and nested classes as well. In the old thread (see github for the link), someone was essentially proposing this idea. He wanted to use a concept of class namespaces, but I like your syntax better.

Note that for me the scope could be reopened only once. In order to avoid that the user can do it on any class it would be better to restrict it to class that have declared it. E.g. we could declare the class friend of itself.

Why the restrictions? Can you demonstrate a case where allowing this feature everywhere would cause a problem? How is this better than using a nested class (see workarounds in my new draft on github for an example). 

//foo.hh
class Foo {
	friend Foo;

};

This again is putting an implementation detail (the fact that we will reopen the class to implement it) in the interface. It seems unnecessary to me. 

//Definition of Foo::pub1()
void Foo::pub1() {
  _priv1();
  _priv3();
}

It is clear that all this would have less sense with modules.

Modules. Who knows what they are or when we will have them. I want something to solve my problems now. Also this can be a step towards modules.

You are right, but you will not have a new feature before 2017 as minor as it can be.

 

Will we even have modules in 2017? There is no guarantee of that. 

[fmatth...@gmail.com]

Everyone, I have written a new draft after taking in all of the feedback. It is available on github.

With regards to method vs member function. It looks like some people use them interchangably while others prefer "method" for virtual functions. I am going to continue using the work method as its already in the name of the proposal.

[mitc...@gmail.com]

On Wednesday, December 11, 2013 3:55:11 AM UTC+1, fmatth...@gmail.com wrote:

[SNIP]

 

Actually using a keyword nicely solves the static dilemna:

private Foo::f1() //private extension method

private static Foo::f2() //static private method

static private Foo::f3() //private extension method with internal linkage

static private static Foo::f4() //static private extension method with internal linkage

Going to think about it more, maybe using the private keyword makes sense.

I think using the private keyword like this is good, but I would prefer to make all PEMs internal linkage by default and use extern or some other keyword in the few cases where you want them to be reachable from other TUs and still not declare them in the class.

Also, making  "private static" and "static private" different things is just too confusing / ugly in my opinion (but that's minor compared to my strong preference towards making PEMs internal linkage by default, thus avoiding this use of the static keyword here).

[inkwizyt...@gmail.com]

You are right, I dont except side effects like calling arbitrary function in function definition.

[Klaim - Joël Lamotte]

On Wed, Dec 11, 2013 at 7:40 AM, <fmatth...@gmail.com> wrote:

Everyone, I have written a new draft after taking in all of the feedback. It is available on github.

I only read new parts but it looks far more complete, well done.

Pure curiosity question: did you thought about the potential impact this proposal could have on some other related proposal, beyond modules?
I'm thinking about the proposal I'm not remembering the name that was allowing calling member functions as non-member functions and vice-versa (like it's allowed in the D language).
I suspect there might be no impact at all, just wondering if some of the proposed syntaxes would close doors to other proposals.

 

With regards to method vs member function. It looks like some people use them interchangably while others prefer "method" for virtual functions. I am going to continue using the work method as its already in the name of the proposal.

Good to know that it's not important for proposal papers.

[Richard Smith]

Sorry this comment is too late for the mailing, but...

Rather than have 'static private static' mean something different from 'static private', have you considered giving private extension methods interrnal linkage by default? (And require 'extern' in the presumably-rare case that they're actually meant to be declared and used in multiple translation units.)

[Klaim - Joël Lamotte]

On Wed, Jan 29, 2014 at 2:26 AM, Richard Smith <ric...@metafoo.co.uk> wrote:

Rather than have 'static private static' mean something different from 'static private', have you considered giving private extension methods interrnal linkage by default? (And require 'extern' in the presumably-rare case that they're actually meant to be declared and used in multiple translation units.)

In the published paper N3863 [1] you can find a section describing alternatives, one of them being giving internal linkage by default (search for "Internal Linkage by Default" or go to the end of the paper).

[1] http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3863.html

[Matt Fioravante]

On Tuesday, January 28, 2014 8:26:37 PM UTC-5, Richard Smith wrote:

Sorry this comment is too late for the mailing, but...

Never too late. The paper will undoubtedly go through many revisions so please by all means discuss your suggestions. Even suggesting an alternative already mentioned in the paper is useful because it shows more people are interested in doing something a certain way.

I'm not opposed to the idea of internal linkage by default for PEMs. That's 99% of the use cases I would them for already anyway.

[Tony V E]

http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3863.html

- I agree with the purpose of the proposal. I write APIs for a living.  I'd prefer to keep out anything that doesn't need to be seen.

- a bug in the paper (?):

//Declare a PEM in the class header file.
//In this case, the effect is the same as declaring it within the class
//definition. Using an extension method here instead of declaring in the
//class definition is merely a stylistic choice.
Foo::_priv2();

//pub2() is inlined and calls _priv1() and _priv2(), so we need to see
//both of them at this point. This can be accomplished by including them
//within the class definition or declaring an extension method as shown
//by the examples of _priv1() and _priv2().
inline int pub2() { return _priv1() + _priv2(); }

That _priv2 needs be to (I think/hope):

private Foo::_priv2();

and the pub2() needs to be declared in the class, and qualified with Foo:: outside the class.

And/Or I don't understand what this part is trying to say.

- In general, I prefer the "reopen Foo" syntax.  I don't like private functions scattered around.  You could still reopen Foo multiple times (and thus have it scattered around) but at least you could recommend against that in a style guideline.  As an example:

void Foo::function()

{

   int x = bar();

}

is bar() a member function of Foo?  We can no longer just look in the class definition to determine that.  Now bar() could be declared "anywhere" (actually anywhere above the current code, including #includes, but practically anywhere).  With reopening Foo, there will at least be a _tendency_ for it to only be reopened at the top of the file and/or a foo_impl.hh file.

Also reopening appears to allow private typedefs, nested classes, static variables, etc.  Which all looks good to me.

Also, a normal function can be declared within any scope. Can a PEM:

void func()

{

    extern int f();

    int x = f();

    private int Foo::blah();

    int y = someFoo.blah(); // of course func() would need to be a friend here

}

or

void Foo::function()

{

    private int blah();  // Foo:: not needed???

 

    int x = blah();

}

- you might entertain using the 'continue' keyword to continue Foo:

class continue Foo

{

   int asdf(); // PEM

};

although using 'private' does help remind us that only private stuff is allowed.

- Internal Linkage by default:  Definitely.

Tony

--

[Tony V E]

On Tue, Jan 28, 2014 at 11:41 PM, Tony V E <tvan...@gmail.com> wrote:

http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3863.html

- I agree with the purpose of the proposal. I write APIs for a living.  I'd prefer to keep out anything that doesn't need to be seen.

- a bug in the paper (?):

//Declare a PEM in the class header file.
//In this case, the effect is the same as declaring it within the class
//definition. Using an extension method here instead of declaring in the
//class definition is merely a stylistic choice.
Foo::_priv2();

//pub2() is inlined and calls _priv1() and _priv2(), so we need to see
//both of them at this point. This can be accomplished by including them
//within the class definition or declaring an extension method as shown
//by the examples of _priv1() and _priv2().
inline int pub2() { return _priv1() + _priv2(); }

That _priv2 needs be to (I think/hope):

private Foo::_priv2();

private int Foo::_priv2();

(missed the return type as well)

[Bengt Gustafsson]

I would favor internal linkage by default. The cases when you split one class into multiple TUs is very low. How about using the now abandoned export keyword instead of extern for the case that you really want to work with multiple TUs for one class. Or I would favor to make it impossible to use private extension methods over TU borders.

The paper seems to have a bug in the nested class workaround example:

X:doWork must send *this rather than x to worker methods!

The table at the top of the paper does not list static data members. It would be possible to add private static data PEM-style. However the value is very limited as it is only a naming difference. However I think it should be in the table and be explained why it is not needed.

[Matthew Fioravante]

On Tuesday, January 28, 2014 11:41:58 PM UTC-5, Tony V E wrote:

That _priv2 needs be to (I think/hope):

private Foo::_priv2();

Thanks, fixed in the latest draft.

 

and the pub2() needs to be declared in the class, and qualified with Foo:: outside the class.

Wow, I'm pretty bad. This is what happens when you rely on inline compilation too much ;)

 

And/Or I don't understand what this part is trying to say.

- In general, I prefer the "reopen Foo" syntax.  I don't like private functions scattered around.  You could still reopen Foo multiple times (and thus have it scattered around) but at least you could recommend against that in a style guideline.  As an example:

void Foo::function()

{

   int x = bar();

}

is bar() a member function of Foo?  We can no longer just look in the class definition to determine that.  Now bar() could be declared "anywhere" (actually anywhere above the current code, including #includes, but practically anywhere).  With reopening Foo, there will at least be a _tendency_ for it to only be reopened at the top of the file and/or a foo_impl.hh file.

Internal linkage by default also sort of solves this problem. All of the PEMs should be found within the same file as the call site.

 

Also reopening appears to allow private typedefs, nested classes, static variables, etc.  Which all looks good to me.

I'm not sure how useful these things would be in practice when you can just create these objects with internal linkage. I'm not against it though, especially if someone can come up with a compelling use case.

 

Also, a normal function can be declared within any scope. Can a PEM:

void func()

{

    extern int f();

    int x = f();

    private int Foo::blah();

    int y = someFoo.blah(); // of course func() would need to be a friend here

}

Sure why not 

or

void Foo::function()

{

    private int blah();  // Foo:: not needed???

 

    int x = blah();

}

Would need to think if removing the Foo:: could cause any conflicts or ambiguities. I don't think it matters much either way.

- you might entertain using the 'continue' keyword to continue Foo:

class continue Foo

{

   int asdf(); // PEM

};

although using 'private' does help remind us that only private stuff is allowed.

I like private better, reusing random keywords is confusing. continue has to do with loops, private is for access control in classes.

 

- Internal Linkage by default:  Definitely.

Looks like this is getting more votes. I might have to shift the next version of the paper in this direction.

On Wednesday, January 29, 2014 10:46:42 AM UTC-5, Bengt Gustafsson wrote:

I would favor internal linkage by default.

And there is another.

 

The cases when you split one class into multiple TUs is very low. How about using the now abandoned export keyword instead of extern for the case that you really want to work with multiple TUs for one class. Or I would favor to make it impossible to use private extension methods over TU borders.

Definitely prefer extern over export because thats what the extern keyword already does. You use extern to give thinks external linkage, why use something else in the case of a PEM?

 

The paper seems to have a bug in the nested class workaround example:

X:doWork must send *this rather than x to worker methods!

Thank you for catching that. Fixed in latest draft.

 

The table at the top of the paper does not list static data members. It would be possible to add private static data PEM-style. However the value is very limited as it is only a naming difference. However I think it should be in the table and be explained why it is not needed.

I also didn't mention typedefs, enums, and nested classes. I'll consider these additions on the next revision.

 

[corn...@google.com]

On Thursday, January 30, 2014 2:53:48 AM UTC+1, Matthew Fioravante wrote:

On Tuesday, January 28, 2014 11:41:58 PM UTC-5, Tony V E wrote:

 

Also reopening appears to allow private typedefs, nested classes, static variables, etc.  Which all looks good to me.

I'm not sure how useful these things would be in practice when you can just create these objects with internal linkage. I'm not against it though, especially if someone can come up with a compelling use case.

Richard Smith has already shown how inner classes plus static members can break encapsulation:

// intrusion.hpp

struct intruder {

  virtual void intrude(fortress& f) = 0;

};

extern intruder* backdoor;

// intrusion.cpp

intruder* backdoor;

private struct fortress::spy : intruder {

  spy() { backdoor = this; }

  virtual void intrude() {fortress& f) {

    f.secret();

  }

};

private static fortress::spy fortress::insertion{};

It seems to me that the rules necessary to prevent this would get quite complicated.

[mikael.s...@gmail.com]

Hi all,

I generally love this idea of being able to extend the private parts of classes privately ;)

Here are my opinions:

I would generally prefer to have internal linkage by default. But that could depend on how well it meshes with the syntax. For the currently proposed syntax (private void Foo::func()), I think that it makes more sense to have internal linkage by default, and use "extern" to change that.

I prefer the "private class" style that was suggested a few times in this thread, because I find the syntax nicer and more intuitive, and I see a lot of potential in being able to define additional private nested types / typedefs, static members (data or functions), and such. I would say that this would be especially beneficial to class templates because I often find that the private parts of class templates can get very cluttered with nested types, typedefs and static members, and I often which I could just separate it away (more on that later below).

Here is an idiom that I have used on a few specific situations for this very same purpose (hiding private functions, and giving them internal linkage). I like to call it the "Cheshire Kitten" idiom, because it's kind of an overhead-free "reduced" Cheshire Cat. It goes something like this:

// in header:

class Foo {

  protected:    // "protected" instead of "private", even though I really want private here

    int value;

  

  public:

    void bar();

};

// in cpp file:

namespace {

class FooImpl : public Foo {

  friend Foo;

  

  void bar_priv() { /* .. some code .. */ };

  //...

};

}

void Foo::bar() {

  static_cast<FooImpl*>(this)->bar_priv();

}

This is obviously not a particularly nice solution (and it's theoretically undefined behaviour because "this" does not actually point to a FooImpl object, but in practice it's safe as long as FooImpl has no data members and there isn't any funny business in the inheritance... so, use with caution). That solution is really about breaking encapsulation (private -> protected) just so that I can hack into the class' internals later on from an anonymous namespace (with FooImpl in it).

The reason for posting this idiom here is also to point out two important things.

The first point is very obvious: this is all about transferability of the private access. In my little idiom, I'm forced to turn private parts into protected parts so that FooImpl can access them (but note that I could do it in other ways too, like a nested accessor class), and then, I have to make Foo a friend of FooImpl (or give public members to FooImpl). A variation of this idiom, with external linkage, involves forward-declaring FooImpl, making it a friend of Foo, and then the rest is the same (define FooImpl to inherit from Foo, and make an "unsafe" static-cast down to it).

So, really, the core of this proposal, to me, is really about providing a private-only bridge across those entities (main class definition, private implementation, and the main class' function bodies). And as long as that bridge is neatly sealed, I see no problems of breaking encapsulation, in fact, quite the opposite, it promotes encapsulation by providing a safe conduit for the access-rights between those entities without having to resort to one hack or another (like my little hacky idiom).

The second point is related to why my idiom is UB. I think this is related to why some people on this thread have pointed out that you might need to make changes to the layout guarantees. My idiom is UB because even though the derived class does not contain any additional data members or any other base class, there is no guarantee that the static-cast is safe. This is because the standard does not really directly specify what comes into the memory layout of a class (except by way of the definition of a "standard-layout" class). Common sense dictates that no compiler on Earth would actually add anything to a class' memory layout for additional non-virtual member functions or for static members. But, technically-speaking, compilers are only required to have this behaviour for standard-layout classes.

So, the interesting thing here is that one way or another, if you open the door to having additional non-virtual member functions that are not present in the class definition, then it implies a requirement that non-virtual member functions do not have any impact on the memory layout of the class. I'm not sure if the C++ standard would absolutely need to make that requirement explicit for your proposal to work, but it's a necessary implication (that you might want to point out in the proposal, in case any compiler-vendor has an issue with that).

So, the reason I like the "private class" version that was discussed here is because this really boils down to being able to create a kind of "ghost class" that is layout-compatible with the main class (like an "overlay") and that has some mechanism for safely transferring access-rights. So, we could have something like this:

class Foo {

  public:

    void bar();

};

private FooImpl : Foo {

  void bar_priv();

  //...

};

void FooImpl::bar_priv() { 

  /* .. some code that has access to Foo's private parts .. */

}

void Foo::bar() {

  this->bar_priv();

}

where FooImpl has a number of restrictions (which will have to be specified in the standard) to ensure that it does not attempt to add anything that would change the class' layout (data members, virtual functions, etc.). And it would both gain access to the private parts of Foo, and be only accessible to Foo (e.g., function bodies of Foo's members). I think that this syntax is nice and intuitive. It can behave as a normal class in terms of linkage (i.e., it has external linkage by default but can be put into an anonymous namespace), and in terms of templates (e.g., FooImpl could be a template, could have specializations, etc..). And finally, you can define multiple such classes, like FooImpl2, FooImpl3..., which might be useful to some (as suggested in reference to the clang::Sema class, which is, btw, a class that belongs in the museum of horrific C++ designs).

Interestingly, when providing an explicit set of restrictions to what can be added to a class without affecting its layout, one could actually do something like this:

public FooExtended : Foo {

  public:

    void another_bar();

    //... additional functions to enhance Foo.

};

which is in no way related to the proposal, but could also be a cool thing. Basically, FooExtended would have the same access-rights as a normal derived class (public and protected parts). And it would be publicly accessible to all as a kind of enhanced interface to Foo, that can be used on any Foo object (i.e., does not need to be created as a FooExtended object). I'm not sure if this feature is really desirable, but it comes essentially for free, because if you put in place all that you need to make the private version work (basic syntax rules, layout-compatibility restrictions, name lookup, access-rights on classes, etc.), the public version is trivial to put in place.

Overall, I also like the "private class" version because it simplifies a lot of the syntax rules, because it eliminates the whole mess with "static private static void Foo::bar_priv();". The "private class" version allows you to just reuse the rules for class definition syntax and the syntax for defining its member functions (outside the class definition) in a way that remains intuitive and harmonious with the creation of normal C++ classes (and class templates). This solution should also make things easier for the compiler implementers.

Another bonus with the "private class" version is that you could specialize parts of a class template, which I know people will absolutely love! What I mean is that you could have something like this:

template <typename T>

struct Foo {

  void bar();

};

template <typename T>

private FooImpl : Foo<T> {

  void bar_priv();

};

template <>

private FooImpl<char> : Foo<char> {

  void bar_priv();

};

template <typename T>

void Foo<T>::bar() {

  this->bar_priv();

}

I think that the ramifications of the above feature might be more than I can currently imagine. But there is no doubt that this kind of a mechanism would be extremely useful and very popular. Currently, specializing parts of a class template is very awkward, if possible at all, and generally involves several awkward layers of helper templates.

And as a final note, I would say that name lookup is probably going to be the most contentious point with compiler vendors. C++ name lookup is already very nasty and consumes a lot of the compilation efforts, and is also a defining structure element for the C++ compilers. I would imagine that amending those rules is not going to be easy, and don't underestimate how much of that will be needed for this proposal. I think that this has to be looked into a lot more. 

This was pretty much all I wanted to say.

BTW, unlike others, I am not concerned with your proposal opening the doors for some hackish ways to break encapsulation. For one thing, C++ already has plenty of loopholes to break encapsulation (some of which are laid out in that gotw post). And then, I sincerely believe that this private-to-private bridge (as I've described it earlier) is fundamentally sound and should provide a fairly safe way to do this. And finally, the alternatives that currently exist (such as my little hacky idiom) often force the programmer to hack their way around the access-rights and therefore leaving some doors open to violations of encapsulation anyways. So, this is a net positive, it provides a safer alternative to hacking the access-rights.

Cheers,

Mikael.

[johannes.g...@gmail.com]

Wouldn't the "it's possible to access private members in a standard-conforming way" holes disappear if private extension functions are internally considered as being templates and thus get the same instantiation rules.

I.e. if

private int MyClass::Extension (int a) { ... }

would be treated as if it were something like

template <class T>
std::enable_if_t<std::is_same<T, MyClass>, int> Extension (T * this, int a) { ... }

with myClass->Extension (2) effectively being translated into Extension (myClass, 2), with Extension being a self-made friend?

As any such call must eventually stem from a non-extension function, private extension functions not anchored in a non-extension function are simply ignored???

Best regards,
Johannes