hidden private member functions

[Viacheslav Usov]

This is about the old and boring problem "private members are part of the class interface".

The idea that I want to propose is the following one: make it possible to omit the declaration of private member functions inside a class definition.

I.e., do something like this

// header file

struct A

{

    void B();

    void C();

private:

    int p;

}

// implementation file

private void A::Q(int x)

{

    p = x;

}

void A::B()

{

    Q(1);

}

void A::C()

{

    Q(2);

}

The private keyword as used above introduces a hidden private member function. Because it is private, it can only be called by other members or friends, so, ultimately, it can only be called by the code that has full access to the class anyway. One cannot just define such a private member to break the encapsulation.

Hidden private members cannot be virtual.

So this is an attempt to make it possible to remove some of the implementation out of the interface. It does not try to remove everything, notably data, that is private. A typical use case would involve some common logic that needs to be called by multiple public members; then the common logic can be put together in such a private member, re-phrasing the same thing, this will make it possible to refactor the implementation without changing the interface files.

It is quite possible that this has been proposed and discussed earlier; please point me to the relevant discussions in that case.

Cheers,

V.

[sval...@gmail.com]

You can already do something like this. Consider:

// A.h
struct A {
    struct B;
    friend struct B;

        void setX();
        int getX() const;

    private:
        int x;
};

// A.cpp
#include "a.h"

struct A::B {
    void foo(A& a) {
        a.x = 4;
    }
};

void A::setX() {
    B().foo(*this);
}

int A::getX() const {
    return x;

[Daniel Boles]

I think this thread is required reading before any change to this can
be discussed:

http://softwareengineering.stackexchange.com/questions/239171/why-do-we
-put-private-member-functions-in-headers

I particularly like the rationale that Sutter gave:

http://softwareengineering.stackexchange.com/a/324450/192238

IMO any proposal for changing this would need to address the various
factors brought up in that thread.

At the very least, this would presumably require something like
modules, in order to ensure that only the person who originally
declared the class can augment it later with 'hidden private members' -
not just any passerby with a cpp file.

[Viacheslav Usov]

On Tue, Dec 6, 2016 at 7:57 PM, Daniel Boles <db0...@gmail.com> wrote:

> IMO any proposal for changing this would need to address the various factors brought up in that thread.

Thanks for the pointer. I'd say the only issue relevant for my proposal is its potential to make name lookup ambiguous. But this one issue make this proposal pretty much unacceptable. Oh well.

Cheers,

V.

[Bengt Gustafsson]

I don't see why such functions could not have a special lookup rule, or a special naming restriction. Possibilties:

1. A hidden private function must not have the same name as any other member function.

2. A hidden private function is only part of the overload set when the call site is in another member function of the same class, and its declaration has been seen (as if it had been a global function).

It may be required to state clearly again that this proposal does not offer any way to break the private protection as these new functions are only callable from other member functions. So while anyone can write one you'd have to change the source code of a non-private member function. If you can do that, well, then you don't need a hidden private function to access the private data members...

[Bengt Gustafsson]

[Vicente J. Botet Escriba]

Hi,

Wondering if modules shouldn't solve the problem?

If it is not the case, I'm not sure we need some syntactic sugar, given we have the suggested and well known private friend class idiom.

In the following example we have in comments what can be done now and with a // ** one possibility for the syntactical sugar version could be.

It uses a variation of the suggested private friend class idiom in this thread

// header file

struct A

{

    void B();

    void C();

private:

    int p;
    //struct implementation;
    // friend implementation;
    implementation; // **

}

I believe it is worth saying that we want to have an implementation, but this could be implicit

// implementation file

//struct A::implementation

implementation struct A // **

{
// A* that;   
//implementation(A* a) : that(a) {}

void Q(int x)

{
    // that->x;

    p = x; // **

}

void R(int x)

{
    Q(x-1); // note that here we don't use that->, as the function is in the implementation friend

}

};

void A::B()

{

    //implementation(this)->Q(1);

    implementation->Q(1); // **

} 

The last use of implementation is needed to ensure that we are not overloading the A public functions with the private implementation functions.
As notice already, an implementation function couldn't be called using the same syntax than a visible function.

Of course the syntax I'm describing here is just one that makes evident the idiom.

I believe the private friend class idiom doesn't introduce too much noise to change the language.


Vicente

[Viacheslav Usov]

On Thu, Dec 8, 2016 at 5:07 AM, Bengt Gustafsson <bengt.gu...@beamways.com> wrote:

> 1. A hidden private function must not have the same name as any other member function.

Unfortunately, we will also need to ensure that its name is different from any other function in all the accessible global and namespace scopes, including those that are introduced after the hidden private function in a given translation unit. And there is no way to force a particular name resolution should the programmer want to.

> 2. A hidden private function is only part of the overload set when the call site is in another member function of the same class, and its declaration has been seen (as if it had been a global function).

I do not think how this changes anything. It is the other member functions where we need name lookup to be unambiguous, because non-members cannot access private functions anyway. So restricting overload resolution to just member functions achieves nothing. Perhaps I misunderstood what you said.

But your response got me thinking, and I have a modified proposal.

The hidden private members shall only be resolvable via a specially qualified name: private::name, where private is the keyword and name is the name of a hidden private member. They are otherwise not part of the name lookup process.

Thus, hidden private members are only accessible to members and friends, and only when those explicitly want to use them.

Cheers,

V.

[Bengt Gustafsson]

> 1. A hidden private function must not have the same name as any other member function.

Unfortunately, we will also need to ensure that its name is different from any other function in all the accessible global and namespace scopes, including those that are introduced after the hidden private function in a given translation unit. And there is no way to force a particular name resolution should the programmer want to.

While I don't really prefer the (1) solution I don't see why your restrictions would be necessary. A method in a class hides global functions of the same name, so whether or not there are other functions with the same name is irrelevant as these hidden private member functions can only be called from within other member functions of the same class (hidden or not).

 

> 2. A hidden private function is only part of the overload set when the call site is in another member function of the same class, and its declaration has been seen (as if it had been a global function).

I do not think how this changes anything. It is the other member functions where we need name lookup to be unambiguous, because non-members cannot access private functions anyway. So restricting overload resolution to just member functions achieves nothing. Perhaps I misunderstood what you said.

Ok. The original concern was that as private member functions are part of the overload set when called from outside the class (causing a protection violation if selected) private member functions can not be tucked away in a cpp file. What I am trying to say is that if we exclude hidden private member functions from the overload set when called from the outside it doesn't matter that they are in a cpp file somewhere. However, to ever be able to call them, they must of course be included in the overload set when called from another member function of the same class. Furthermore, I think that it is more logical to only let them participate in overload resolution after their declaration in the same way that global functions are.

But your response got me thinking, and I have a modified proposal.

The hidden private members shall only be resolvable via a specially qualified name: private::name, where private is the keyword and name is the name of a hidden private member. They are otherwise not part of the name lookup process.

I think you are overestimating the problem here: The case that there is overloading at all between hidden and non-hidden member functions is going to be rare. And as these functions are very private such problems can easily be solved by changing the name of the hidden private method. To add a requirement to prefix the call site adds unneccessary noise.

Vicente complained about not being able to explicitly select a hidden or non-hidden method by some scoping mechanism. I think this is no worse than the fact that you can't select between two overloaded functions in general without casting the parameters. A hidden member function and a non-hidden member function are both member functions of the same class according to the original proposal and my overload-set related rule. Vicente's own example is very hard to follow but it seems to NOT think of the hidden private member functions as part of the same scope, which makes the feature much closer to the current work around (which seems to be the point he wants to prove).

 

Thus, hidden private members are only accessible to members and friends, and only when those explicitly want to use them.

I don't see the point or logic in the "explicitly want to use them" part. Isn't this implicit by ordering  the declarations so that the hidden private method declaration preceeds the calling method?

[ subtopic: Should it be allowed to forward declare private hidden methods: My reply is yes - this is parallel to free function declarations, and required to allow them to be mutually recursive. ]

Cheers,

V.

[Viacheslav Usov]

On Sat, Dec 10, 2016 at 12:53 AM, Bengt Gustafsson <bengt.gu...@beamways.com> wrote:

> While I don't really prefer the (1) solution I don't see why your restrictions would be necessary. A method in a class hides global functions of the same name, so whether or not there are other functions with the same name is irrelevant as these hidden private member functions can only be called from within other member functions of the same class (hidden or not).

> Ok. The original concern was that as private member functions are part of the overload set when called from outside the class (causing a protection violation if selected) private member functions can not be tucked away in a cpp file. What I am trying to say is that if we exclude hidden private member functions from the overload set when called from the outside it doesn't matter that they are in a cpp file somewhere. However, to ever be able to call them, they must of course be included in the overload set when called from another member function of the same class. Furthermore, I think that it is more logical to only let them participate in overload resolution after their declaration in the same way that global functions are.

I may still be misinterpreting you.

My concern is not that the hidden private members can be defined in a rogue cpp file and somehow break encapsulation. As I stated in the original message, they are only callable by members (and friends), so that's not a concern at all. I have not tried to address that again in follow-up messages.

What I did try to address is the situation when one translation unit defines hidden private members, while another does not; moreover, let's postulate that when they are defined they hide some global functions, or they overload some other members. Then the lookup for those names (unless modified per my second proposal) will work differently between those two TUs, potentially breaking the program silently.

Different name lookup, while possible for non-member function even today, is not possible for member functions. The latter is a strong and useful guarantee afforded by member functions, breaking it would be very bad. Worse still, this can even break currently valid code, which knows nothing about hidden private members. I'm pretty sure this cannot ever pass through the committee.

Note also that hidden member functions could, in principle, be injected by a rogue include file, even though I believe it is far more likely to happen through error rather than malice.

Which is why I think that the opt-in through private:: is necessary. Yes, this makes things a little more verbose than I would have liked; but this is balanced by not having to decorate "implementation details" in some other way, which is frequently the case in the real-world code.

Cheers,

V.

[Bengt Gustafsson]

My concern is not that the hidden private members can be defined in a rogue cpp file and somehow break encapsulation. As I stated in the original message, they are only callable by members (and friends), so that's not a concern at all. I have not tried to address that again in follow-up messages.

Sure. 

What I did try to address is the situation when one translation unit defines hidden private members, while another does not; moreover, let's postulate that when they are defined they hide some global functions, or they overload some other members. Then the lookup for those names (unless modified per my second proposal) will work differently between those two TUs, potentially breaking the program silently.

For this to happen you must subdivide implementation of methods in the same class into different TUs which is very uncommon. I think you are chasing very unlikely cases and in the process make your own proposal harder to use.  

Different name lookup, while possible for non-member function even today, is not possible for member functions. The latter is a strong and useful guarantee afforded by member functions, breaking it would be very bad. Worse still, this can even break currently valid code, which knows nothing about hidden private members. I'm pretty sure this cannot ever pass through the committee.

I don't understand this statement, or I find it contradictory. My suggestion avoids the breakage of currently valid code by stating that the hidden private membder functions are not partaking in overload resolution unless their declarations have been seen. Thus having them can't break any (client) code that hasn't seen them.

The entire aim of the proposal is to allow hiding helper methods in the cpp file implementing the class's regular methods. Now you seem to worry about what would happen if you put hidden member functions between the class head and the client using the class. But as the client is not one of the member functions of the class it doesn't see hidden member functions even if they are wtritten (for mysterious reasons) in a header file read by the client code. This includes if the client is a subclass of the class having the hidden private method, as these methods are private, not protected.

This rule allows hidden private methods to be used to implement common functionality between inlined methods (when written below the class head) without disrupting the rule that client code can rely on the class head to contain "all you need to know about the class".

 

Note also that hidden member functions could, in principle, be injected by a rogue include file, even though I believe it is far more likely to happen through error rather than malice.

How? Ok, if you tamper with the header file and then recompile the cpp file implementing the class methods. But in this case you obviously have access to the cpp file so if you want to tamper it is much easier to just change the code.

There is no way to protect against reaching in to the private data members for instance, given that you take the liberty to change the header file. If you don't change the header file I don't see how rouge code could be injected except in the rare case that the original implementer opted to implement inline methods in a totally separate header file than the class head, which is very impractical and hence uncommon. As you can always change the header file to do what you want C++ can never protect from malificent usage of classes you have the header file for.

 

Which is why I think that the opt-in through private:: is necessary. Yes, this makes things a little more verbose than I would have liked; but this is balanced by not having to decorate "implementation details" in some other way, which is frequently the case in the real-world code.

I think you are being bizarrely cautious here. Obviously an implementor would not use hidden private methods if s/he doesn't know how they work. If you know how they work you probably start out by not selecting the same name as a regular member function (which makes the rest of the debate moot) but if you do select the same name that's probably because you want the overloading to take place. I think it would be hard to find a use case for this, but of course there is a slight risk of inadvertent overloading, but there are so many error sources on this level already, I don't see how this is any worse. I also find the rule that "when the declaration of a hidden private method is parsed it is added to the overload set for its name when called from member functions of the same class" is logical and easy to understand and implement.