Fixing the private method issue

[fmatth...@gmail.com]

There is one major wart in C++ class design, and that is with private member functions required to be in the class definition.

This has a number of problems:

1) Changing the private method's signature or adding/removing private methods requires everyone including the header to recompile. This is a huge problem for large projects with long recompilation times.

2) file static / anonymous namespace definitions in the .cc file cannot be used in the private method's signature. Anything used in the signature must be at least forward declared in the header, adding more symbol pollution.

3) For shared library interfaces, the private methods are extra unnecessary symbols that have to be managed.

4) Private method signatures or even the existence of private methods can depend on the underlying implementation. If the class has multiple implementations (e.g. different platforms), #defines and other conditional compilation mechanisms are required in the header file.

5) Its just bad encapsulation. Callers don't need to know anything about the functions which implement the class behavior.

The only private declarations that should be required in the header are declarations required by the compiler. I believe all of those are:

1) Private data members (sizeof())

2) Private virtual methods (for inheriting)

3) Private non-virtual methods called by inline functions.

One way to do this would be to extend the friend feature. We could define friend functions within the body of member functions. It might look like this:

//foo.hh

class Foo {
  public:

    void doWork();

  private:

    int _f;

}

//foo.cc

static void doWorkHelper(Foo* f) {

  doSomethingWith(f->_f);
};

void Foo::doWork() {
  friend void doWorkHelper(Foo*);

  doWorkHelper(this);

}

This has potential for abuse of course, but it would finally allow us to limit the list of declarations in the class definition to the bare minimum required by the compiler. When it comes to defining interfaces, less is always more.

One other use of this feature could be to add a backdoor for unit tests.

Thoughts?

[Billy O'Neal]

> One other use of this feature could be to add a backdoor for unit tests.

 

Unit tests that test non-public data are bad (brittle) unit tests. Break up those classes! :)

 

For most cases where these bits matter you can define a static non-member non-friend function in a file somewhere, and have your member function implementations call these non-members.

 

Another mitigation is to put your "private member functions" into another class, and declare that class a friend. You still leak the one symbol, but that's comparatively minor:

 

http://ideone.com/K9m0Fz

 

#include <iostream>

class PublicClass
{
 friend class PublicClassImpl;
 int idx;
public:
 PublicClass() : idx (42) {}
 void DoWork();
};

 

// In the implementation file:
class PublicClassImpl
{
public:
    static void DoWorkImpl(PublicClass& target);
};

 

void PublicClass::DoWork()
{
 PublicClassImpl::DoWorkImpl(*this);
}

 

void PublicClassImpl::DoWorkImpl(PublicClass& target)
{
 std::cout << "Index is currently " << target.idx++ << std::endl;
}

 

// Example Use:

int main() {
 PublicClass pc;
 pc.DoWork();
 pc.DoWork();
 return 0;
}

 

Billy O'Neal

https://github.com/BillyONeal/

http://stackoverflow.com/users/82320/billy-oneal

Malware Response Instructor - BleepingComputer.com

--
 
---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.

[Cassio Neri]

Some suggestions similarly to Billy O'Neal's are given here:

https://groups.google.com/forum/?fromgroups#!searchin/comp.lang.c$2B$2B.moderated/Moving$20private$20functions$20to$20opaque$20inner/comp.lang.c++.moderated/GJyXgeEtAOA/PwXSG_ZGeGgJ

Cheers,
Cassio.

[Ville Voutilainen]

I have also used a solution where I define a public struct for the members of the class

and have the members in a private instance of that struct, and then pass a reference
to that struct to implementation-file-scope 'private' functions. This way the only

private member functions that need to be in the class definition are the ones that
are virtual.

So,

1) Changing the private method's signature or adding/removing private methods requires everyone including the header to recompile. This is a huge problem for large projects with long recompilation times.

- not a problem anymore

2) file static / anonymous namespace definitions in the .cc file cannot be used in the private method's signature. Anything used in the signature must be at least forward declared in the header, adding more symbol pollution.

- ditto

3) For shared library interfaces, the private methods are extra unnecessary symbols that have to be managed.

- I can use implementation-specific visibility settings for this, so not a problem

4) Private method signatures or even the existence of private methods can depend on the underlying implementation. If the class has multiple implementations (e.g. different platforms), #defines and other conditional compilation mechanisms are required in the header file.

- as for 1&2, not a problem

5) Its just bad encapsulation. Callers don't need to know anything about the functions which implement the class behavior.

- ditto

The tediousness of such a solution is seemingly not big enough to really want a language
change, according to my experience.

I haven't used an opaque array blob from which the implementation-functions allocate space
to the real members. I have considered it a couple of times, but never bothered using it. And it

doesn't help much for cases where the size of the blob must increase.

The Lakosbook (http://www.amazon.com/Large-Scale-Software-Design-John-Lakos/dp/0201633620)

provides further insight for physical design and insulation.

[mitc...@gmail.com]

Hello,

   I've been thinking about a new proposal for a "class implementation namespace", where I initially thought to only allow member function implementations that were already declared, but reading this proposal I realized that allowing new private member function declarations would solve your problem. It would look something like this using your example:

//foo.hh

class Foo {
  public:

    void doWork();

  private:

    int _f;

}

//foo.cc:

class Foo namespace {

// optional forward declaraction:

  void doWrokHelper();  // implicitly private

// definition of doWorkHelper() private member function declared above:

  void doWorkHelper() {

     doSomethingWith(_f);

  }

  

// definition of doWork() public member function declared in Foo.hh:

  void doWork() {

     doWorkHelper();

  }

} // end class Foo namespace (the "implementation namespace")

In my opinion this "class implementation namespace" would be a cleaner solution to your issue, and would also have additional benefits:

  - template class member function implementations could be separated from the main class declaration without having to resort to ugly repetitions

  - member functions returning nested classes could be written the same way as declared (this is a common problem for new learners according to my experience)

  - in general, the "implementation" part would be syntactically similar to the declaration part, without unnecessary repetition of the class name, etc

I'll try to write a detailed proposal next week (I first wanted to implement this concept in clang, but I guess it's better to get out the idea first...)

cheers,

mitch

[Billy O'Neal]

Private member *data* does need to be declared in the class definition so that callers know what size to lay out for the object. Private member *functions* do not.

Billy O'Neal

https://github.com/BillyONeal/

http://stackoverflow.com/users/82320/billy-oneal

Malware Response Instructor - BleepingComputer.com

On Sun, Nov 3, 2013 at 1:39 PM, <shado...@gmail.com> wrote:

Maybe not intuitive, but this is by design. For performance reasons, the compiler needs to know the layout of a struct, and it can only do that if the definition is visible. Any workarounds require an indirection AFAIK, as does implementing this as standard behavior. If you want hiding and abi compatibility use the PIMPL idiom for private data. But realize that it adds an extra indirection.

-Tim

On Wednesday, October 30, 2013 4:39:17 AM UTC+1, fmatth...@gmail.com wrote:

--

[Philipp Stephani]

Adding a virtual member functions to a class without other virtual member functions does change the class layout. (I guess classes whose virtual member functions are all private don't occur in practice though.)

2013/11/4 Billy O'Neal <billy...@gmail.com>

[Billy O'Neal]

Actually that's a good point -- most of the iostreams classes' virtual functions are all private, for example. The functions added in a single translation unit would have to be non-virtual.

Billy O'Neal

https://github.com/BillyONeal/

http://stackoverflow.com/users/82320/billy-oneal

Malware Response Instructor - BleepingComputer.com

[Philipp Stephani]

Their destructor is public. However, there can exist classes with protected nonvirtual destructors (that allow dynamic dispatch for calling but not for destruction).

I'm not sure whether a restriction should be made to only allow private nonvirtual functions outside the class definition -- a rule could be added that the class layout may not be different for any nonzero number of virtual functions, and the compiler could signal an error if a virtual function is to be added to a class whose definition doesn't contain any virtual functions.

In general I guess modules can solve this problem in a more generic way.

[Philipp Stephani]

2013/11/4 Philipp Stephani <p.ste...@gmail.com>

I'm not sure whether a restriction should be made to only allow private nonvirtual functions outside the class definition -- a rule could be added that the class layout may not be different for any nonzero number of virtual functions, and the compiler could signal an error if a virtual function is to be added to a class whose definition doesn't contain any virtual functions.

This is wrong: As already pointed out by the OP, all virtual functions can be overridden and are therefore part of the interface, regardless of their visibility. 

[Thiago Macieira]

No, they cannot.

Overriding a virtual function may change the ABI. *Any* virtual declaration
must be visible to all callers.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[Ville Voutilainen]

On 4 November 2013 04:36, Thiago Macieira <thi...@macieira.org> wrote:

On segunda-feira, 4 de novembro de 2013 03:09:00, Philipp Stephani wrote:
> 2013/11/4 Philipp Stephani <p.ste...@gmail.com>
>
> > I'm not sure whether a restriction should be made to only allow private
> > nonvirtual functions outside the class definition -- a rule could be added
> > that the class layout may not be different for any nonzero number of
> > virtual functions, and the compiler could signal an error if a virtual
> > function is to be added to a class whose definition doesn't contain any
> > virtual functions.
>
> This is wrong: As already pointed out by the OP, all virtual functions can
> be overridden and are therefore part of the interface, regardless of their
> visibility.

No, they cannot.

They cannot? As in, virtual functions cannot be overridden? Are you thinking
about final here?
 

Overriding a virtual function may change the ABI. *Any* virtual declaration
must be visible to all callers.

Same question, I guess.

[Thiago Macieira]

On segunda-feira, 4 de novembro de 2013 06:02:39, Ville Voutilainen wrote:
> They cannot? As in, virtual functions cannot be overridden? Are you thinking
> about final here?
>
>
> > Overriding a virtual function may change the ABI. *Any* virtual
> > declaration
> > must be visible to all callers.

> Same question, I guess.

Ok, after rereading what I posted, looks like I might not have answered the
right post. So let me start over:

This thread is about adding methods to a class without having it in the main
class { } body, which is visible to all users. I support that idea.

However, that must be entirely restricted to non-virtual methods.

Adding a virtual method, like everyone who has done any binary compatibility
work knows, is not possible. That includes overrides. Any and all virtual
methods present in a given class -- whether new or overrides -- must be
declared in the class body and visible to all users.

[Ville Voutilainen]

On 4 November 2013 06:15, Thiago Macieira <thi...@macieira.org> wrote:

On segunda-feira, 4 de novembro de 2013 06:02:39, Ville Voutilainen wrote:
> They cannot? As in, virtual functions cannot be overridden? Are you thinking
> about final here?
>
>
> > Overriding a virtual function may change the ABI. *Any* virtual
> > declaration
> > must be visible to all callers.

> Same question, I guess.

Ok, after rereading what I posted, looks like I might not have answered the
right post. So let me start over:

This thread is about adding methods to a class without having it in the main
class { } body, which is visible to all users. I support that idea.

However, that must be entirely restricted to non-virtual methods.

Adding a virtual method, like everyone who has done any binary compatibility
work knows, is not possible. That includes overrides. Any and all virtual
methods present in a given class -- whether new or overrides -- must be
declared in the class body and visible to all users.

I still have trouble grokking the "that includes overrides" part. How does adding
an override change the ABI?

[Billy O'Neal]

It is overridden in one translation unit, but not in another translation unit. That means the vtbl in each translation unit differs, which leads to ODR violations. (This is assuming virtual functions are implemented in terms of vtbls, but all implementations of which I am aware implement things that way)

Billy O'Neal

https://github.com/BillyONeal/

http://stackoverflow.com/users/82320/billy-oneal

Malware Response Instructor - BleepingComputer.com

[Thiago Macieira]

On segunda-feira, 4 de novembro de 2013 06:41:12, Ville Voutilainen wrote:
> I still have trouble grokking the "that includes overrides" part. How does
> adding an override change the ABI?

See these two exceptions to overriding virtuals
http://techbase.kde.org/Policies/Binary_Compatibility_Examples#Override_a_virtual_with_a_covariant_return_with_different_top_address
http://techbase.kde.org/Policies/Binary_Compatibility_Examples#Override_a_virtual_that_doesn.27t_come_from_a_primary_base

These two overrides are actually new virtuals in disguise under the portable
IA-64 C++ ABI used by GCC and Clang. Since those two cases are possible, we
have to take into consideration the possibility that other overrides are
actually new virtuals under other ABIs.

That means we must treat all overrides as if they were new virtuals for the
purpose of defining the standard.

[Ville Voutilainen]

On 4 November 2013 07:28, Thiago Macieira <thi...@macieira.org> wrote:

On segunda-feira, 4 de novembro de 2013 06:41:12, Ville Voutilainen wrote:
> I still have trouble grokking the "that includes overrides" part. How does
> adding an override change the ABI?

See these two exceptions to overriding virtuals
http://techbase.kde.org/Policies/Binary_Compatibility_Examples#Override_a_virtual_with_a_covariant_return_with_different_top_address
http://techbase.kde.org/Policies/Binary_Compatibility_Examples#Override_a_virtual_that_doesn.27t_come_from_a_primary_base

These two overrides are actually new virtuals in disguise under the portable
IA-64 C++ ABI used by GCC and Clang. Since those two cases are possible, we
have to take into consideration the possibility that other overrides are
actually new virtuals under other ABIs.

That means we must treat all overrides as if they were new virtuals for the
purpose of defining the standard.

Thanks for the explanation. Also, as Billy pointed out, it would be rather weird
to have any attempts to add "virtual extension methods" in one TU and not have
them in another. It would be a step towards object inheritance from class inheritance,
and I don't think we want to take such steps.

I still don't think the (specification/implementation/teaching/etc.) cost of extension
methods is palatable. There are well-known solutions to the problem that are good
enough that the benefit of adding extension methods to the language is unconvincing
to me.

[Thiago Macieira]

On segunda-feira, 4 de novembro de 2013 07:46:44, Ville Voutilainen wrote:
> I still don't think the (specification/implementation/teaching/etc.) cost of
> extension methods is palatable. There are well-known solutions to the
> problem that are good enough that the benefit of adding extension methods
> to the language is unconvincing to me.

Well, the one thing I'd want is to have a real static (as in file-local)
method, to control my exports.

I know all the techniques. But I am forced to use compiler extensions in order
to control the list of exports from my TUs and libraries, to degrading the
linkers' performances due to way too many symbols.

[Billy O'Neal]

I'm not sure "extension methods" are related here -- at least, anywhere else I've seen that concept they've not been able to access private details of a class.

Billy O'Neal

https://github.com/BillyONeal/

http://stackoverflow.com/users/82320/billy-oneal

Malware Response Instructor - BleepingComputer.com

--

[Olaf van der Spek]

On Monday, November 4, 2013 2:05:15 AM UTC+1, Billy O'Neal wrote:

Private member *data* does need to be declared in the class definition so that callers know what size to lay out for the object. Private member *functions* do not.

Not entirely true. In general, callers don't need to know the layout. The layout is (only?) required when accessing data members, the size is only required when creating an object on the stack.

Not requiring private data members in the header would be nice too IMO.

[Marshall Clow]

On Nov 4, 2013, at 5:18 AM, Olaf van der Spek <olafv...@gmail.com> wrote:

On Monday, November 4, 2013 2:05:15 AM UTC+1, Billy O'Neal wrote:

Private member *data* does need to be declared in the class definition so that callers know what size to lay out for the object. Private member *functions* do not.

Not entirely true. In general, callers don't need to know the layout. The layout is (only?) required when accessing data members, the size is only required when creating an object on the stack.

Counterexample:

struct Foo;

Foo * f = new Foo;

This calls operator new ( size_t ) to allocate memory.

How much memory should be allocated?

Not requiring private data members in the header would be nice too IMO.

-- Marshall

Marshall Clow     Idio Software   <mailto:mclow...@gmail.com>

A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait).
        -- Yu Suzuki

[Olaf van der Spek]

On Mon, Nov 4, 2013 at 3:01 PM, Marshall Clow <mclow...@gmail.com> wrote:
> Not entirely true. In general, callers don't need to know the layout. The
> layout is (only?) required when accessing data members, the size is only
> required when creating an object on the stack.
>
>
> Counterexample:
> struct Foo;
> Foo * f = new Foo;
>
> This calls operator new ( size_t ) to allocate memory.
> How much memory should be allocated?

One could use a factory function
Or some (automatic) get_size() function could be generated

[Thiago Macieira]

On segunda-feira, 4 de novembro de 2013 15:14:56, Olaf van der Spek wrote:
> One could use a factory function
> Or some (automatic) get_size() function could be generated

Let's not go there, please.

Instead, if this is a desired feature, let's just make it so the compiler
knows that it's a partial declaration. Until the declaration is full, the type
is incomplete and its size is not known.

[Olaf van der Spek]

On Mon, Nov 4, 2013 at 5:22 PM, Thiago Macieira <thi...@macieira.org> wrote:
> On segunda-feira, 4 de novembro de 2013 15:14:56, Olaf van der Spek wrote:
>> One could use a factory function
>> Or some (automatic) get_size() function could be generated
>
> Let's not go there, please.
>
> Instead, if this is a desired feature, let's just make it so the compiler
> knows that it's a partial declaration. Until the declaration is full, the type
> is incomplete and its size is not known.

The idea is to move all private parts out of the header, avoiding
rebuilds on changes to private parts.


--
Olaf

[xavi]

Allowing the declaration of private member data outside of the header would also make inheritance or composition with this class impossible, so any use of this class would need to be through a pointer. In this case, what would be the advantage of this with respect to using the pimpl idiom?

2013/11/4 Olaf van der Spek <olafv...@gmail.com>

[Thiago Macieira]

On segunda-feira, 4 de novembro de 2013 18:11:01, xavi wrote:
> Allowing the declaration of private member data outside of the header would
> also make inheritance or composition with this class impossible, so any use
> of this class would need to be through a pointer. In this case, what would
> be the advantage of this with respect to using the pimpl idiom?

It would simplify the code for the implementation. A good pimpl solution
requires all public functions to be mirrored in the private so that you don't
get a double indirection to the data (it also improves code generation because
the private class isn't exported).

This would be basically a way for C++ place member functions in opaque types.

[Billy O'Neal]

>The idea is to move all private parts out of the header, avoiding rebuilds on changes to private parts.

 

That isn't possible in a language with value semantics everywhere. Java and C# allow this by forcing all interactions with a class to occur through pointers / references, and by generating code at runtime for correct inheritance behavior. Those aren't acceptable tradeoffs for a language like C++.

Billy O'Neal

https://github.com/BillyONeal/

http://stackoverflow.com/users/82320/billy-oneal

Malware Response Instructor - BleepingComputer.com

[Richard Smith]

On Sun, Nov 3, 2013 at 9:10 AM, <mitc...@gmail.com> wrote:

Hello,

   I've been thinking about a new proposal for a "class implementation namespace", where I initially thought to only allow member function implementations that were already declared, but reading this proposal I realized that allowing new private member function declarations would solve your problem. It would look something like this using your example:

//foo.hh

class Foo {
  public:

    void doWork();

  private:

    int _f;

}

//foo.cc:

class Foo namespace {

"class X namespace" looks a lot like "allow me to violate access control here, please".

I think you should require the class to opt into this somehow (perhaps by requiring a forward declaration of this inside the class body, or requiring every such namespace to contain a member of the class).

But since we're brainstorming here, we can get most of the benefit here with extension methods alone:

// foo.h

class Foo {

public:

  void doWork();

private:

  int _f;

  friend struct FooImpl;

};

// foo.cc

struct FooImpl {

  static void doWorkHelper(Foo *this) {

    doSomethingWith(_f);

  }

};

void Foo::doWork() {

  doWorkHelper();

}

--

[xavi]

As long as you store the pimpl pointer in a local variable (so that the compiler knows that the pointer doesn't change no matter what you call), the compiler can generate exactly the same code for accessing members of pimpl as it would for accessing members of this (which is a pointer, after all). As for the private class being exported, that's the kind of details that the standard usually leaves to implementations, and a good compiler with link-time optimization would be able to remove these symbols anyway.

2013/11/4 Thiago Macieira <thi...@macieira.org>

[Thiago Macieira]

On segunda-feira, 4 de novembro de 2013 19:14:48, xavi wrote:
> As long as you store the pimpl pointer in a local variable (so that the
> compiler knows that the pointer doesn't change no matter what you call),
> the compiler can generate exactly the same code for accessing members of
> pimpl as it would for accessing members of this (which is a pointer, after
> all). As for the private class being exported, that's the kind of details
> that the standard usually leaves to implementations, and a good compiler
> with link-time optimization would be able to remove these symbols anyway.

It's the kind of detail that the standard can make easier for the compilers,
so the compilers can generate better code under more use-cases.

Link-time optimisation isn't enough in some cases, isn't possible in some
others, and isn't state-of-the-art yet.

Maybe the solution will come with modules, when we can finally have a
standardised syntax for "this method is called by other modules" versus "this
method is never called externally"

[Zhihao Yuan]

On Mon, Nov 4, 2013 at 1:01 PM, Richard Smith <ric...@metafoo.co.uk> wrote:
>> //foo.cc:
>>
>> class Foo namespace {
>
>
> "class X namespace" looks a lot like "allow me to violate access control
> here, please".

I don't think so. This scope, afaics, is private, with contents safe
to be added without changing the ABI.

> But since we're brainstorming here, we can get most of the benefit here with
> extension methods alone:
>
> // foo.h
> class Foo {
> public:
> void doWork();
> private:
> int _f;
> friend struct FooImpl;
> };
>
> // foo.cc
> struct FooImpl {
> static void doWorkHelper(Foo *this) {
> doSomethingWith(_f);
> }
> };
>
> void Foo::doWork() {
> doWorkHelper();
> }

I see two problems here:

1. Program is written for human to read. This approach, with
functions taking pointer to objects, basically reverted C++ to C;

2. Friend is flawed: what you need is to grant accessibility to
a set of functions, but what you do is to grant accessibility to
another class scope. I read this as another "solution from
implementation's view" like using private to make a class
non-copyable.

There should be more simple, more obvious answer to the problem
raise in this thread.

--
Zhihao Yuan, ID lichray
The best way to predict the future is to invent it.
___________________________________________________
4BSD -- http://4bsd.biz/

[Richard Smith]

On Mon, Nov 4, 2013 at 12:36 PM, Zhihao Yuan <z...@miator.net> wrote:

On Mon, Nov 4, 2013 at 1:01 PM, Richard Smith <ric...@metafoo.co.uk> wrote:
>> //foo.cc:
>>
>> class Foo namespace {
>
>
> "class X namespace" looks a lot like "allow me to violate access control
> here, please".

I don't think so.  This scope, afaics, is private, with contents safe
to be added without changing the ABI.

If you can add friends to a 'class namespace', you can trivially use it to violate access control (and get access to private members from some third party TU). You can get the same effect even without allowing friends in a 'class namespace', but you need to work a little harder. For instance:

struct Launderer {

  virtual int get(Foo *p) = 0;

} *launderer;

class Foo namespace {

  struct FooLaunderer : Launderer {

    FooLaunderer() { launderer = this; }

    int get(Foo *p) { return p->_f; }

  } static theLaunderer;

}

Foo::FooLaunderer Foo::theLaunderer;

int evil_get_member(Foo *p) { return launderer->get(p); }

You can get the same effect without a static data member in the 'class namespace', but again it takes a little more work. And you can do the same thing without even using a member class (using a local class instead of a member function instead). So: this subtly breaks access control.

*shrug* It doesn't seem like a big problem to me, especially not once we have modules. I don't think it warrants a big heavy language feature of its own.

[Zhihao Yuan]

On Mon, Nov 4, 2013 at 5:17 PM, Richard Smith <ric...@metafoo.co.uk> wrote:
>> I don't think so. This scope, afaics, is private, with contents safe
>> to be added without changing the ABI.
>

> struct Launderer {
> virtual int get(Foo *p) = 0;
> } *launderer;
>
> class Foo namespace {
> struct FooLaunderer : Launderer {
> FooLaunderer() { launderer = this; }
> int get(Foo *p) { return p->_f; }
> } static theLaunderer;

I see. class namespace can be re-opened.

>> 1. Program is written for human to read. This approach, with
>> functions taking pointer to objects, basically reverted C++ to C;
>>
>> 2. Friend is flawed: what you need is to grant accessibility to
>> a set of functions, but what you do is to grant accessibility to
>> another class scope. I read this as another "solution from
>> implementation's view" like using private to make a class
>> non-copyable.
>>
>> There should be more simple, more obvious answer to the problem
>> raise in this thread.
>
>
> *shrug* It doesn't seem like a big problem to me, especially not once we
> have modules. I don't think it warrants a big heavy language feature of its
> own.

I'm not +1 for class namespace the specific idea. Can you elaborate
on how to use modules to solve this?

[mitc...@gmail.com]

On Monday, November 4, 2013 11:17:24 PM UTC+1, Richard Smith wrote:

On Mon, Nov 4, 2013 at 12:36 PM, Zhihao Yuan <z...@miator.net> wrote:

On Mon, Nov 4, 2013 at 1:01 PM, Richard Smith <ric...@metafoo.co.uk> wrote:
>> //foo.cc:
>>
>> class Foo namespace {
>
>
> "class X namespace" looks a lot like "allow me to violate access control
> here, please".

I don't think so.  This scope, afaics, is private, with contents safe
to be added without changing the ABI.

If you can add friends to a 'class namespace', you can trivially use it to violate access control (and get access to private members from some third party TU). You can get the same effect even without allowing friends in a 'class namespace', but you need to work a little harder. For instance:

struct Launderer {

  virtual int get(Foo *p) = 0;

} *launderer;

class Foo namespace {

  struct FooLaunderer : Launderer {

    FooLaunderer() { launderer = this; }

    int get(Foo *p) { return p->_f; }

  } static theLaunderer;

}

The above code would not be valid in "class namespace", there is no "free-floating this pointer" just because you're in the "class namespace". As I wrote in my original post, the primary motivation for class-namespace is to allow the implementation of already declared members to be less verbose / more uniform, the new idea sparkled by this thread is to allow declaration / definition of yet undeclared _private_ member functions. I don't think they could be used to circumvent access control in any way.

Basically, the following would be allowed in class-namespace:

- using declarations

- typedefs

- definition of previously declared members / member functions

- declaration / definition of new _private_ member functions or _private_ static member functions (these should probably be "special" regarding linkage, they should be TU bound and not exported, but I'm not (yet) well-versed with this area to express my ideas here correctly, sorry)

- nested class-namespaces for implementation of previously declared nested classes

Does this make sense?

[fmatth...@gmail.com]

On Wednesday, October 30, 2013 12:12:10 AM UTC-4, Billy O'Neal wrote:
>
> Unit tests that test non-public data are bad (brittle) unit tests. Break up those classes! :)


99% of the time I agree, however there are some cases where it is
unavoidable. One example is optimizations. For example, perhaps your
class is a template that is optimized for POD types. The effect may
not actually be visible from the public interface, in which case you
may need to dig into the private interface to ensure the optimization
triggers for all of the right cases and works as expected.

Right now this is impossible to do without doing horrible things like
adding extra public methods, friend declarations, or making everything
protected and inheriting. All of these are horrible.

On Wednesday, October 30, 2013 11:02:01 AM UTC-4, Ville Voutilainen wrote:
>
>
> I have also used a solution where I define a public struct for the members of the class
> and have the members in a private instance of that struct, and then pass a reference
> to that struct to implementation-file-scope 'private' functions. This way the only
> private member functions that need to be in the class definition are the ones that
> are virtual.


This works but I'm not sure I like it as it exposes another public symbol.

The other trick is the private nested class with static methods. No
symbols are exposed to the user but this is still a coupling of the
class definition with the private member implementations.
Whoever implements the private methods, regardless of how many C files
are used is restricted to using this one nested class.

On Sunday, November 3, 2013 8:44:45 PM UTC-5, Philipp Stephani wrote:
>
> Adding a virtual member functions to a class without other virtual member functions does change the class layout. (I guess classes whose virtual member functions are all private don't occur in practice though.)


Actually all virtual functions with the exception of the destructor
should almost always be private. This is the non-virtual idiom.
http://www.gotw.ca/publications/mill18.htm

On Sunday, November 3, 2013 12:10:17 PM UTC-5, mitc...@gmail.com wrote:
>
>    I've been thinking about a new proposal for a "class implementation namespace",

This is pretty interesting. One potential abuse is that if anyone
wants to just avoid your public interface they can open the namespace
and hack in something. I could see horrible last minute bug fixes
being implemented this way. Still, access control is just a tool for
allowing us to write better interfaces, if people want to do stupid
things we don't need to be responsible for them.

The friend idea I proposed limits the access control to the
implementation of member functions. Only someone writing the actual
member functions can extend the access control out to another
function.

On Sunday, November 3, 2013 9:06:39 PM UTC-5, Philipp Stephani wrote:
>
> I'm not sure whether a restriction should be made to only allow private nonvirtual functions outside the class definition -- a rule could be added that the class layout may not be different for any nonzero number of virtual functions, and the compiler could signal an error if a virtual function is to be added to a class whose definition doesn't contain any virtual functions.

As suggested by others, virtual functions must be in the class
definition. Every part of the interface which is exposed to users must
be in the class definition, it should not be spread out and makes
little sense to do so. More on that later.

On Monday, November 4, 2013 12:56:11 AM UTC-5, Thiago Macieira wrote:
>
> Well, the one thing I'd want is to have a real static (as in file-local)
> method, to control my exports.
>
> I know all the techniques. But I am forced to use compiler extensions in order
> to control the list of exports from my TUs and libraries, to degrading the
> linkers' performances due to way too many symbols.

I was under the impression that static and anonymous namespace
functions don't export symbols even in single object files. In fact
the function may be removed entirely or even sliced up into callable
pieces if the compiler decides to inline it everywehre.

With regards to the general problem of symbol visibility, that's
probably something that would probably involve [attributes] and is
outside of this discussion entirely.

On Monday, November 4, 2013 8:18:22 AM UTC-5, Olaf van der Spek wrote:
>
> Not entirely true. In general, callers don't need to know the layout. The layout is (only?) required when accessing data members, the size is only required when creating an object on the stack.

> Not requiring private data members in the header would be nice too IMO.

But how would you compute the size without knowing the layout? You
need to know the layout for padding. Also, what real use is knowing
the size without knowing the layout. As soon as you change a data
member your callers will need to recompile anyway as the size will
change.

 On Monday, November 4, 2013 2:41:35 PM UTC-5, Thiago Macieira wrote:
>
> Maybe the solution will come with modules, when we can finally have a
> standardised syntax for "this method is called by other modules" versus "this
> method is never called externally"

Maybe, maybe not. What are modules? When will they become available?
We have no idea. I'd like a workable solution to this problem now.


I'd like to focus the discussion a bit. Lets agree on some invariants.
The following must remain in the class definition.
* data members with any access control.
* virtual methods with any access control.
* public and protected non-virtual methods, they are part of the interface.
* private non-virtual methods called by inline functions, for obvious reasons.

Maybe someone can come up with a good reason to break one of these but
I don't want to discuss it here. If you're passionate about that then
please start a new thread.

Not only are there hard implementation reasons for these, restricting
the external interface to a single point, namely the class interface
is an extremely powerful constraint in terms of usability and
readability. I don't need to chase down multiple header files, source
files, and documentation, I just need to see one class definition and
it has everything I need to know as a possible user of the class
(using directly or inheriting). The only thing I don't need to know
about is how many functions are used to implement its behavior, and
those are the non-virtual private methods we are discussing.

So far we have 2 suggestions, declaring friends in function bodies and
having a class namespace.

Here's another, simply define allow a syntax to define new private
methods outside of the class definition. These methods should *always*
be private and therefore only callable by other class methods. What we
don't want is to allow people to start writing public or protected
extension methods, making interfaces impossibly complicated. This
allows easy extensibility and also keeps the interface constrained to
whats in the definition.

class Foo {
  public:
  Foo(int i);
};

//explicitly define a new private method which is not in the class body
explicit void Foo::_foo() {
}

//explicitly define a new private constructor, callable by other constructors
explicit Foo::Foo() {
}

//Define the public constructor which calls the explicit private one
using delegation
explicit Foo::Foo(int i) : Foo() {
}

I like this better than the namespace proposal, because with the
namespace, you have this problem:
class Foo namespace {

//Hundreds of lines of code

//Wait, is this an extension method or a normal function? I have to look for the namespace tags.
void foo();

};
Also if the only thing that's going in your namespace is function
definitions, do you really need to create a scope for that? I always
prefer to use static for file local functions instead of anonymous
namespaces. I use anonymous namespaces for file local global variables
and class definitions.

The namespace idea however would allow you to define other things such as private nested types and typedefs.

I also like it better than the friend proposal, because adding friend
to already cluttered function bodies is hard to read and rather
intelligent. Its also not obvious looking at a function knowing
whether or not something else has declared it as a friend.

[fmatth...@gmail.com]

On Wednesday, November 6, 2013 3:34:45 AM UTC-5, mitc...@gmail.com wrote:

Basically, the following would be allowed in class-namespace:

- using declarations

- typedefs

- definition of previously declared members / member functions

- declaration / definition of new _private_ member functions or _private_ static member functions (these should probably be "special" regarding linkage, they should be TU bound and not exported, but I'm not (yet) well-versed with this area to express my ideas here correctly, sorry)

- nested class-namespaces for implementation of previously declared nested classes

Does this make sense?

I'd recommend making everything in the class namespace private to the class, and by that I mean nothing outside of the class can use any of its  declarations. It would be as if they were declared directly in the classes private section. This prevents anyone from actually modifying the external class interface and avoids the "break access control here" anti-pattern that surely some people would try to use. In this way the class namespace is merely a tool for making the class implementation easier to write.

[mitc...@gmail.com]

That's exactly what I had in mind, but probably failed to communicate clearly.  Everything in class-namespace is private to the class. 

[Olaf van der Spek]

On Wed, Nov 6, 2013 at 1:21 PM, <fmatth...@gmail.com> wrote:
> On Monday, November 4, 2013 8:18:22 AM UTC-5, Olaf van der Spek wrote:
>>
>> Not entirely true. In general, callers don't need to know the layout. The
>> layout is (only?) required when accessing data members, the size is only
>> required when creating an object on the stack.
>> Not requiring private data members in the header would be nice too IMO.
>
>
> But how would you compute the size without knowing the layout? You
> need to know the layout for padding. Also, what real use is knowing
> the size without knowing the layout. As soon as you change a data
> member your callers will need to recompile anyway as the size will
> change.

If you pass by reference / pointer there's no need to know size, is there?

[billy...@gmail.com]

If you pass by reference there's no need for the type to be complete either.

Sent from a touchscreen. Please excuse the brevity and tpyos.

> wrote:
> On Monday, November 4, 2013 8:18:22 AM UTC-5, Olaf van der Spek wrote:
>>
>> Not entirely true. In general, callers don't need to know the layout. The
>> layout is (only?) required when accessing data members, the size is only
>> required when creating an object on the stack.
>> Not requiring private data members in the header would be nice too IMO.
>
>
> But how would you compute the size without knowing the layout? You
> need to know the layout for padding. Also, what real use is knowing
> the size without knowing the layout. As soon as you change a data
> member your callers will need to recompile anyway as the size will
> change.

If you pass by reference / pointer there's no need to know size, is there?

-- 

--- 
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.

[billy...@gmail.com]

>One example is optimizations. For example, perhaps your

class is a template that is optimized for POD types

If you can't see any difference big enough to reliably unit test, the performance gains aren't worth the additional complexity.

Sent from a touchscreen. Please excuse the brevity and tpyos.

----- Reply message -----
From: fmatth...@gmail.com
To: <std-pr...@isocpp.org>
Subject: [std-proposals] Re: Fixing the private method issue

On Monday, November 4, 2013 8:18:22 AM UTC-5, Olaf van der Spek wrote:
>
> Not entirely true. In general, callers don't need to know the layout. The layout is (only?) required when accessing data members, the size is only required when creating an object on the stack.
> Not requiring private data members in the header would be nice too IMO.


But how would you compute the size without knowing the layout? You
need to know the layout for padding. Also, what real use is knowing
the size without knowing the layout. As soon as you change a data
member your callers will need to recompile anyway as the size will
change.

--

[Olaf van der Spek]

On Wed, Nov 6, 2013 at 5:25 PM, billy...@gmail.com
<billy...@gmail.com> wrote:
> If you pass by reference there's no need for the type to be complete either.

Member functions can't be called on an incomplete type

[Billy O'Neal]

I'm confused -- first you want to be able to add private non-virtual member functions to a class inside of a single translation unit, which seems reasonable. Now you're proposing something else -- completely separating the public interface of the class from its representation. I don't think that'd be near as useful -- the number of translation units which mention the type, call public member functions of the type, but don't try to copy or otherwise interact with it are going to be extremely small. Plus, such a feature would be far harder to specify given the way the language currently works.

Billy O'Neal

https://github.com/BillyONeal/

http://stackoverflow.com/users/82320/billy-oneal

Malware Response Instructor - BleepingComputer.com

[Olaf van der Spek]

On Wed, Nov 6, 2013 at 7:01 PM, Billy O'Neal <billy...@gmail.com> wrote:
> I'm confused -- first you want to be able to add private non-virtual member
> functions to a class inside of a single translation unit, which seems
> reasonable. Now you're proposing something else -- completely separating the
> public interface of the class from its representation. I don't think that'd
> be near as useful -- the number of translation units which mention the type,
> call public member functions of the type, but don't try to copy or otherwise
> interact with it are going to be extremely small.

What do you mean by otherwise interacting?
Non-copyable classes like iostream et are used quite useful and are
used quite a lot, without any need for copying them.

[Billy O'Neal]

>Non-copyable classes [...] are used quite useful and are used quite a lot, without any need for copying them.

Yes, they are. However, one always has to balance how often something is used with the scope of the change in the language proposed. I don't think this pattern is near common enough to merit having to define a completely new class declaration syntax for people to learn. Of course, I'm not on the committee or anything like that so it isn't like my opinion matters.

 

>iostream

The iostreams classes are templates, and as a result need even public member definitions available in each translation unit, so that's not a great example.

 

 

Billy O'Neal

https://github.com/BillyONeal/

http://stackoverflow.com/users/82320/billy-oneal

Malware Response Instructor - BleepingComputer.com

[fmatth...@gmail.com]

I think I prefer my idea of being able to just define private methods outside of the class body, explicit keyword or not.

//foo.hh

class Foo {

public:

  int foo();
};

//private extension method declaration in the header file

explicit int Foo::_bar();

//inline private extension method

explicit inline int Foo::_baz() { return 5; }

//inline public method calls private extension method

inline int Foo::foo() { return _bar; }

//foo.cc

//private extension method which only appears in the file.

explicit int Foo::_bar_impl() {

  return 42;

}

//private extension method definition

explicit int Foo::_bar() {

  return _baz_impl();

}

With the namespace proposal, you get to define extra things like nested classes and typedefs, but is there any reason you would actually ever need to do this? The only time I find myself defining private nest classes or typedefs is when I'm using them to define my private class data members. In those situations, they would have to be in the class definition anyway.

If you want to define helper classes and typedefs, you can simply make them file local which is even better, or if they need to be shared between translation units, use a private header file.

[mitc...@gmail.com]

On Tuesday, November 12, 2013 4:58:46 AM UTC+1, fmatth...@gmail.com wrote:

I think I prefer my idea of being able to just define private methods outside of the class body, explicit keyword or not.

 I also came to the conclusion that mixing my class-namespace proposal with this is probably not the best idea. There should be a way to define "hidden" private member functions without class-namespace, too.  The only common ground with the two proposals is that if the new "hidden" private member declaration will turn out to be similar to a normal member definition (with an extra keyword like 'explicit' or 'private'), then it should be possible to define the same in class-namespace (without the extra class scope specifiers and such).

One issue that crossed my mind when thinking about "breaking" an existing class: it should not be possible to add a "hidden" private member that adds an overload to an existing declared private member.  I don't know if there's a similar constraint elsewhere in the standard, or if it'd be completely new...

[fmatth...@gmail.com]

One additonal wrinkle is how to define static private extension methods:

//Is this file local or a static class method?

static explicit int Foo::sfoo() { /* ... */ }

Perhaps this syntax?

//File local private extension method

static explicit int Foo::f1() { /* ... */ }

//private extension static class method

explicit int Foo::f2() static { /* ... */ }

//File local private extension static class method

static explicit int Foo::f3() static { /* ... */ }

On Thursday, November 14, 2013 6:45:17 AM UTC-5, mitc...@gmail.com wrote:

On Tuesday, November 12, 2013 4:58:46 AM UTC+1, fmatth...@gmail.com wrote:

I think I prefer my idea of being able to just define private methods outside of the class body, explicit keyword or not.

 I also came to the conclusion that mixing my class-namespace proposal with this is probably not the best idea. There should be a way to define "hidden" private member functions without class-namespace, too.  The only common ground with the two proposals is that if the new "hidden" private member declaration will turn out to be similar to a normal member definition (with an extra keyword like 'explicit' or 'private'), then it should be possible to define the same in class-namespace (without the extra class scope specifiers and such).

We can write 2 competing proposals. Either one I'd be happy with to improve compile times and improve encapsulation by hiding more implementation details from the user.

 

One issue that crossed my mind when thinking about "breaking" an existing class: it should not be possible to add a "hidden" private member that adds an overload to an existing declared private member.  I don't know if there's a similar constraint elsewhere in the standard, or if it'd be completely new...

I think its probably ok to do this. You can define overloads for normal functions anywhere. Why not private member functions. The only taboo in my mind is breaking encapsulation, which is what would happen if we allowed extension of public, protected, data members, and/or virtual. 

[mitc...@gmail.com]

On Thursday, November 14, 2013 2:26:42 PM UTC+1, fmatth...@gmail.com wrote:

[SNIP]

We can write 2 competing proposals. Either one I'd be happy with to improve compile times and improve encapsulation by hiding more implementation details from the user.

The original goal  / scope of my (would be) class-namespace proposal is to allow writing the implementation of class members with a less verbose / more uniform (with regards to the declaration) syntax, shoehorning these hidden-private members into it doesn't seem like a good idea to me any more, so if anything solid comes out from this proposal, I'll address it in the class-namespace proposal, too, if appropriate, but I won't introduce this feature there.

 

 

One issue that crossed my mind when thinking about "breaking" an existing class: it should not be possible to add a "hidden" private member that adds an overload to an existing declared private member.  I don't know if there's a similar constraint elsewhere in the standard, or if it'd be completely new...

I think its probably ok to do this. You can define overloads for normal functions anywhere. Why not private member functions. The only taboo in my mind is breaking encapsulation, which is what would happen if we allowed extension of public, protected, data members, and/or virtual. 

But you _can_ break encapsulation by introducing private overloads, consider this example:

class Foo {

   int bar_;

   // ...

   int getBarImpl() const;

public:

   int getBar() { return getBarImpl(); }

};

// evil.cc:

// non-const override of Foo::getBarImpl():

explicit int Foo::getBarImpl() { bar_ = 42;  /* direct access to private member... */ }

Foo f;

f.getBar(); // calls our evil override getBarImpl()

[fmatth...@gmail.com]

Actually I don't even think this hole will work as intended. Consider this example:

http://gcc.godbolt.org/#%7B%22version%22%3A3%2C%22filterAsm%22%3A%7B%22labels%22%3Atrue%2C%22directives%22%3Atrue%2C%22commentOnly%22%3Atrue%7D%2C%22compilers%22%3A%5B%7B%22source%22%3A%22%5Cn%5Cn%5Cnint%20foo(const%20int%26%20i)%20%7B%5Cn%20%20return%201%3B%5Cn%7D%5Cn%5Cninline%20int%20bar()%20%7B%5Cn%20%20int%20x%20%3D%202%3B%5Cn%20%20return%20foo(x)%3B%5Cn%7D%5Cn%5Cntemplate%20%3Ctypename%20T%3E%5Cninline%20int%20baz()%20%7B%5Cn%20%20T%20x%20%3D%202%3B%5Cn%20%20return%20foo(x)%3B%5Cn%7D%5Cn%5Cnint%20foo(int%26%20i)%20%7B%5Cn%20%20return%203%3B%5Cn%7D%5Cn%5Cnint%20main()%20%7B%5Cn%20%20%2F%2Funcomment%20to%20see%20result%5Cn%20%20%2F%2Freturn%20bar()%3B%5Cn%20%20%2F%2Freturn%20baz%3Cint%3E()%3B%5Cn%20%20return%20baz%3Cconst%20int%3E()%3B%5Cn%7D%22%2C%22compiler%22%3A%22%2Fusr%2Fbin%2Fg%2B%2B-4.8%22%2C%22options%22%3A%22-O3%20-march%3Dnative%20-std%3Dc%2B%2B11%20%22%7D%5D%7D 

[Bengt Gustafsson]

I think it has been stated before in this thread that methods called by inline methods declared in the class head must be in the class head. This should mean that only overloads visible at the point of definition of the calling inline method's body should be considered whenever that method is instantiated or inlined. In the example this would mean that getBar() could never call the non-const version of getBarImpl().

Maybe this was what you meant, "fmatth"?

When it comes to extension methods vs. opening a class scope to implement methods I think these should be two orthogonal features. I also suggest to use another syntax than "class namespaec Foo" to reopening a class. Like this:

Foo:: {

    // Implement methods declared in class head:

    void Foo(int) {}    // A ctor

    int GetPi() { return 3; }   // Some other method

    // Combination of both features adds an extension method without stating Foo::

    explicit void ExtraMethod() { cout << "Extra" << endl; }

}

// A "regular" extension method.

explicit void Foo::AdditionalExtra()

{

}

It could also be considered to replace the explicit keyword with "private" as it makes it clear that the method is private. The drawback could be that it becomes a bit hard to explain that if you put "private" in front of a method declared in the class head it is an error while if you do it in front of another method signature it is ok. Anyway, I think I would personally prefer "private":

private void Foo::AnotherExtension()

{

}

Another drawback could be that some would ask why public and protected are not allowed in the same way...