Auto-Pimpl by partial
Izzy Coding
https://groups.google.com/a/isocpp.org/forum/m/?pli=1#!topic/std-discussion/2KNNorMv6Is
--- Start of OP ----
Not sure if anything like this has been discussed before.
I have been having a dark time with a recent project where one of my requirements is about reduction of dependencies. I also think this idea might enable simple and more declarative code for most use cases.
Ok, so, most of you have probably had the need for a class that does one thing and internally requires the use of other external libraries. While also following the rule of KISS and DRY, this requires private functions that have parameters of external types.
Example:
{{
class thing {
public:
thing(....);
string DoSomething(....);
private:
string PrivateFromExternal(externalType a, ....);
externalType PrivateToExternal(string b, ....);
};
}}
Someone who inherits this class automatically now has a dependency on externalType and thus would require linking against the library binaries.
However, with the development of meta-classes and other mechanisms, I am wondering if now is the time to allow something that has been in other languages for a while.
The ability of partial classes.
The way I see this working is that only when a compile sees the partial keyword (could be different if "partial" is not acceptable), then the definition should be open for expansion later in the translation unit.
This way in one header file I can expose the minimal public definition of the class, and then later define the private internal methods. This would also allow me to chain header files to allow multiple levels of definition.
The actual implementation could do this by making each subsequent partial class a class that inherits from the previous type and then define an inline using for the partial type to the name required for the class.
For example:
first.h
{{
#include <string>;
partial class Thing {
public:
Thing(....);
void DoSomething(....);
string GetSomething(....);
// compiler would add =>
//private: baseThing _internal;
// where baseThing is the next partial/final class definition
// compiler would then copy public and protected methods into this class and make them forward to the internal type
};
// compiler would define a using definition here from the mashed function name to Thing for use by other classes
}}
second.h
{{
#include "first.h"
#include <map>;
partial class Thing {
protected:
Thing(std::map<string,string> a, ....);
string ToJson(void) const;
void FromJson(string);
// compiler would add =>
//private: baseThing _internal;
// where baseThing is the next partial/final class definition
// compiler would then copy public and protected methods into this class and make them forward to the internal type
};
// compiler would re-define using definition here from the mashed function name to Thing for use by other classes
}}
thing.cpp
{{
#include "second.h";
#include <external_lib>;
final class Thing {
private:
externalType ToET(....);
string FromET(....);
string CallServiceUsingET(externalType, ....);
}
/* All Implementation Details Here */
}}
Something like this would allow me to use Thing in various ways depending on my requirements and locality to the type. Thing, is a single entity to reason about regardless of what level of access I require. Also by only including first.h in another class I can use Thing without having to have my code depend and link with external_lib.
This facility could also use the meta-classes proposal so that each partial definition could be a meta-class to further reduce verbosity in its definition, but would also ensure that we have the cleanest code possible, that is easy to reason about.
This I think would be handy, but wanted to see if there was anything I had forgotten before I try and write a proposal.
All feedback welcome (even if it is to point out how stupid this idea is). I know this may be difficult to actually implement, because it requires forward/future definition dependency. But, I think this could be solved using reflection somehow. I am not an expert, so I expect Simone to say "oh, that could be done like this ....)" or "can't do that because of ....".
Hope this idea sparks some interest as I think this could be a useful feature of the language (or library if that is where it is better suited).
---- End of OP ----
Summary of previous response:
Could be solved by:
- Template functions and then provide specialisation in cpp
- Actual Pimpl boilerplate library code using smart_ptrs
- Modules
My reply to these:
Yes these could solve the direct problem of hiding the initial problem of leaking dependencies. Also using free functions in the cpp file could do the same hiding.
However, I don't believe any of these options are particularly clear or easy to understand at a glance.
One of the main aims is to provide a simple way to enable automatic pimpl and also when used with meta-classes could provide a lot of flexibility and streamline boilerplate code.
I have not really created many examples here as this was quickly thrown together to get some initial feedback. I may try to write this up in full and provide a lot more useful points to this other than just the auto-pimpl idea. I would be interested in any other opinions and ideas.
Thiago Macieira
> #include "first.h"
> #include <map>;
>
> partial class Thing {
> protected:
> Thing(std::map<string,string> a, ....);
> string ToJson(void) const;
> void FromJson(string);
>
> // compiler would add =>
> //private: baseThing _internal;
> // where baseThing is the next partial/final class definition
> // compiler would then copy public and protected methods into this class and
> make them forward to the internal type };
meaning that sizeof(Thing) is also unknown?
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
Izzy Coding
If you numbered the things (giving them unique names), then each Thing would have an internal unique_ptr<Thing..> where .. would be the id of the next "partial" in the "stack". This can be done as if partial is specified on a class then there would have to be at least a final class. Therefore one could forward declare Thing(n+1) and then just keep a pointer to it.
Whether that is right or not I don't know. But it would be nice to have some clear and easy syntax to allow me to both define pimpl-classes and also allow me flexibility to share the pimpl definition with multiple classes that maybe only differ on a few methods and their implementation.
I know this is partially covered by meta-classes, but that does not allow me to extend definitions across multiple files without boilerplate code that looks ugly and is sometime difficult to reason about unless your an evangelist. Also I would prefer to be able to keep my public interface clean and have my source code readable by even a newbie cpper.
Thiago Macieira
> What I meant by baseThing is:
> If you numbered the things (giving them unique names), then each Thing would
> have an internal unique_ptr<Thing..> where .. would be the id of the next
> "partial" in the "stack". This can be done as if partial is specified on a
> class then there would have to be at least a final class. Therefore one
> could forward declare Thing(n+1) and then just keep a pointer to it.
But that's not what I understand a partial class to be. You're chaining
pointers and introducing overhead. The incomplete definition solution achieves
the same without that overhead, but at the expense of making it impossible to
instantiate the type until it's complete.
I can see advantages and disadvantages for both cases.
> Whether that is right or not I don't know. But it would be nice to have some
> clear and easy syntax to allow me to both define pimpl-classes and also
> allow me flexibility to share the pimpl definition with multiple classes
> that maybe only differ on a few methods and their implementation.
(private) part of different first (public) parts?
> I know this is partially covered by meta-classes, but that does not allow me
> to extend definitions across multiple files without boilerplate code that
> looks ugly and is sometime difficult to reason about unless your an
> evangelist. Also I would prefer to be able to keep my public interface
> clean and have my source code readable by even a newbie cpper.
Izzy Coding
Given an object of type Thing,
And I include first.h,
When I use the instance in my code,
Then I expect to have access to the methods defined in first.h on the instance of Thing,
Given an object of type Thing,
And I include second.h (that could have public methods and data members defined),
When I use the instance in my code,
Then I expect to have access to the methods defined by first.h,
And I expect to have access to the methods defined by second.h
This may look just like interfaces and inheritance, but is more about having to avoid the ugliness and extra code (that is very easy to get wrong) required to manually do pimpl or method implementation hiding.
Ultimately both cases would point to a fully defined and constructed instance of Thing, but would act similar to a class hierarchy where the user has static_cast<RelevantThingLevel>. The memory layout could be implemented much the same as a multi-layer class hierarchy, but as a single object. The "pointer" to Thing(n+1) could be removed during compilation and maybe have structs defined for each high level partial which would use explicit conversion functions that just forward method and data calls to the fully fledged object. So, I don't believe there would be any real overhead as it would just be a "view" into the full object at run-time.
Also there could be an optimiser opportunity to "merge" multiple partial definitions if nothing depends directly on the higher level partial class?
If it was possible to delay naming the partial class type, then it could possibly enable the compiler.n() calls used in meta-classes proposal to enable one high level definition to be used by 2 differing classes. This would not mean they are related in any way, but would just reduce the number of time I would have to specify the same things on multiple classes.
Thiago Macieira
> What I meant about referencing from multiple classes is that,
>
> Given an object of type Thing,
> And I include first.h,
> When I use the instance in my code,
> Then I expect to have access to the methods defined in first.h on the
> instance of Thing,
of another structure?
> This may look just like interfaces and inheritance, but is more about having
> to avoid the ugliness and extra code (that is very easy to get wrong)
> required to manually do pimpl or method implementation hiding.
different.
The biggest challenge with pimpl is that the secondary object needs to be
allocated in the heap. You haven't explained if this is part of your design
and, if so, how it gets allocated and deallocated.
> Ultimately both cases would point to a fully defined and constructed
> instance of Thing, but would act similar to a class hierarchy where the
> user has static_cast<RelevantThingLevel>. The memory layout could be
> implemented much the same as a multi-layer class hierarchy, but as a single
> object. The "pointer" to Thing(n+1) could be removed during compilation and
> maybe have structs defined for each high level partial which would use
> explicit conversion functions that just forward method and data calls to
> the fully fledged object. So, I don't believe there would be any real
> overhead as it would just be a "view" into the full object at run-time.
Izzy Coding
Inheritance would require multiple class definitions spread across multiple sets of .h and .cpp files (or if you like to dump multiple things in a single file you could do it as 2 files, but I doubt many people do this). Also This allows creation of FirstThing independent of FullThing.
Partial just tells the compiler that I don't care what the full size of Thing is, I just want access to the view I do have a definition for. These can only be created in full (requiring include of full definition chain so allocation can happen as normal), but seen and used as a partial view of that full type as required. A FirstThing is a FullThing, but where I only care about the size of memory that is defined in first.h. I hope this makes sense.
This allows me to spread the definition of a single type across multiple files so that I can keep my includes as minimal as possible and means the using class/function only has to know about what it is going to use.
Also when used with meta-classes each partial could be defined as a different meta-class which would include only the required compiler transforms available to each level of partial. Which would allow the separation of concerns for each set of functionality in a Thing.
Thiago Macieira
> I'm not sure exactly how the implementation would work as I have not sat
> down to fully flesh it out yet. But, I would hope and strive to ensure zero
> overhead where possible.
at any point in the instantiation.
That means your feature request is no different than inheritance, with the
possible exception of making the partial types non-instantiable without
requiring a pure virtual method inside. Of course, you could achieve the same
by way of a protected, default constructor (or private if you want to ensure
only you can derive from it).
The only visible difference I can think of would be the typeid() and reflection
of the objects.
> Inheritance would require multiple class definitions spread across multiple
> sets of .h and .cpp files (or if you like to dump multiple things in a
> single file you could do it as 2 files, but I doubt many people do this).
> Also This allows creation of FirstThing independent of FullThing.
> Partial just tells the compiler that I don't care what the full size of
> Thing is, I just want access to the view I do have a definition for. These
> can only be created in full (requiring include of full definition chain so
> allocation can happen as normal), but seen and used as a partial view of
> that full type as required. A FirstThing is a FullThing, but where I only
> care about the size of memory that is defined in first.h. I hope this makes
> sense.
>
> This allows me to spread the definition of a single type across multiple
> files so that I can keep my includes as minimal as possible and means the
> using class/function only has to know about what it is going to use.
> Also when used with meta-classes each partial could be defined as a
> different meta-class which would include only the required compiler
> transforms available to each level of partial. Which would allow the
> separation of concerns for each set of functionality in a Thing.
Matthew Woehlke
> Example:
> {{
> class thing {
> public:
> thing(....);
> string DoSomething(....);
> private:
> string PrivateFromExternal(externalType a, ....);
> externalType PrivateToExternal(string b, ....);
> };
> }}
>
> Someone who inherits this class automatically now has a dependency on
> externalType and thus would require linking against the library
> binaries.
the headers, and only if you were unable to forward declare the types.)
--
Matthew
Bengt Gustafsson
Izzy Coding
> the headers, and only if you were unable to forward declare the types.)
How do you manage that then?
Every time I have used a class that defines the usage of an external type I have had to "include" that libraries includes directory and "link" to its binary witching my build tool and compiler config.
What this is trying to do is reduce the requirement of having to know the full size of a Thing when using it. I should only need to care about the size of the partial Thing that I care about. If "constructing" a new Thing then that would require knowledge of the full type (hence the idea of "final" to mark the end of the chain).
Matthew Woehlke
>> Uh... no? Only if they are actually using those types. (They only need
>> the headers, and only if you were unable to forward declare the types.)
>
> How do you manage that then?
struct foo
{
void x(undef);
undef y();
};
struct bar : foo
{
};
int main()
{
bar x;
return 0;
}
The implementations of `foo::x`/`foo::y` need the full definition of
`undef`, but derived classes (e.g. `bar`) don't. The above compiles and
links just fine, at least on GCC. (Yes, even though the methods are
public, because nothing in the above actually *calls* them. Since in
your case they were private, obviously nothing outside of `thing` or
friends will call them.)
--
Matthew