Design patterns with T*, where T is never defined

[Maciej Sobczak]

In the latest release of "Large Scale C++", Lakos describes a wrapping technique that involves passing pointers between objects of unrelated classes (say, class A and class B), where the pointer is of a third type (say, class T) that is only declared, but does not need to be ever defined. The pointer value is internally reinterpret_casted to whatever is needed, but this detail is not visible to the user of these classes.
So:

class T; // declared, but never defined

class A
{
public:
T * getT(); // reinterpret_cast<T *>(...)
};

class B
{
public:
void foo(T * p); // reinterpret_cast<something *>(p)
};

Without this idiom, the pointer would be passed around in the form of void*, but the use of class T adds some type safety, enables overloading, etc.

Lakos described this pattern as a way to hide inheritance between types that are supposed to be hidden behind a wrapper facade (like for example T being a "base class" for A, and B operating on that "base"), but my question is whether you have seen a wider use of this pattern.

The type safety is a defining added value of this pattern and it might have a wider applicability. Can you share your experiences?

--
Maciej Sobczak * http://www.inspirel.com

[Alf P. Steinbach]

On 22.03.2020 23:14, Maciej Sobczak wrote:
> In the latest release of "Large Scale C++", Lakos describes a
> wrapping technique that involves passing pointers between objects of
> unrelated classes (say, class A and class B), where the pointer is of
> a third type (say, class T) that is only declared, but does not need
> to be ever defined. The pointer value is internally
> reinterpret_casted to whatever is needed, but this detail is not
> visible to the user of these classes. So:
>
> class T; // declared, but never defined
>
> class A
> {
> public:
> T * getT(); // reinterpret_cast<T *>(...)
> };
>
> class B
> {
> public:
> void foo(T * p); // reinterpret_cast<something *>(p)
> };
>
> Without this idiom, the pointer would be passed around in the form of
> void*, but the use of class T adds some type safety, enables
> overloading, etc.
>
> Lakos described this pattern as a way to hide inheritance between
> types that are supposed to be hidden behind a wrapper facade (like
> for example T being a "base class" for A, and B operating on that
> "base"), but my question is whether you have seen a wider use of this
> pattern.

Not /exactly/ the same, but the handle types in the Win32 API match.

> The type safety is a defining added value of this pattern and it
> might have a wider applicability. Can you share your experiences?

A main problem in the Win32 API is when there are several conceptual
handle types that during their lifetimes, between creation and
destruction, act as the same kind, like a pointer to base class. In
particular HDC (handle to device context), if I recall correctly.
Depending on how it was created such handle must be destroyed by a
corresponding function, but the type does not encode that.


- Alf

[Melzzzzz]

On 2020-03-22, Maciej Sobczak <see.my....@gmail.com> wrote:
> In the latest release of "Large Scale C++", Lakos describes a wrapping
> technique that involves passing pointers between objects of unrelated
> classes (say, class A and class B), where the pointer is of a third
> type (say, class T) that is only declared, but does not need to be
> ever defined. The pointer value is internally reinterpret_casted to
> whatever is needed, but this detail is not visible to the user of
> these classes. So:
>
> class T; // declared, but never defined
>
> class A { public: T * getT(); // reinterpret_cast<T *>(...) };
>
> class B { public: void foo(T * p); // reinterpret_cast<something *>(p)
> };
>
> Without this idiom, the pointer would be passed around in the form of
> void*, but the use of class T adds some type safety, enables
> overloading, etc.

What type safety? It is worse as you don't need reinterpret cast
in case of passing void*....

>


--
press any key to continue or any other to quit...
U ničemu ja ne uživam kao u svom statusu INVALIDA -- Zli Zec
Svi smo svedoci - oko 3 godine intenzivne propagande je dovoljno da jedan narod poludi -- Zli Zec
Na divljem zapadu i nije bilo tako puno nasilja, upravo zato jer su svi
bili naoruzani. -- Mladen Gogala

[Melzzzzz]

Problem with win32 API is because it uses DWORD integer and recently,
DWORD_PTR so you must use reinterpret_cast :(
>
>
> - Alf

[Paavo Helde]

On 23.03.2020 7:16, Melzzzzz wrote:
> On 2020-03-22, Maciej Sobczak <see.my....@gmail.com> wrote:
>> In the latest release of "Large Scale C++", Lakos describes a wrapping
>> technique that involves passing pointers between objects of unrelated
>> classes (say, class A and class B), where the pointer is of a third
>> type (say, class T) that is only declared, but does not need to be
>> ever defined. The pointer value is internally reinterpret_casted to
>> whatever is needed, but this detail is not visible to the user of
>> these classes. So:
>>
>> class T; // declared, but never defined
>>
>> class A { public: T * getT(); // reinterpret_cast<T *>(...) };
>>
>> class B { public: void foo(T * p); // reinterpret_cast<something *>(p)
>> };
>>
>> Without this idiom, the pointer would be passed around in the form of
>> void*, but the use of class T adds some type safety, enables
>> overloading, etc.
>
> What type safety? It is worse as you don't need reinterpret cast
> in case of passing void*....

Double static_cast to void* and back to T* is equivalent to a
reinterpret_cast.

Actually, with the "T*" interfaces, one does not need any
reinterpret_casts in the C++ implementation code which sees the class
definitions. Upcasting in a class hierarchy is implicit and downcasting
can be done by a single static_cast.

reinterpret_cast would be needed in the client code, which does not see
class definitions. However, reinterpret_cast is very under-specified and
may easily fail (especially in case of multiple or virtual inheritance,
but I'm pretty sure the results are formally undefined or
implementation-specific also in simpler scenarios).

So I would ban such casts in the client side code and provide explicit
conversion functions instead if the client code really needs to cast
between different derived class pointer types. In short,
reinterpret_cast is not a proper way to implement anything, I'm a bit
surprised if Lakos really recommends it.

[Melzzzzz]

On 2020-03-23, Paavo Helde <myfir...@osa.pri.ee> wrote:
> On 23.03.2020 7:16, Melzzzzz wrote:
>> On 2020-03-22, Maciej Sobczak <see.my....@gmail.com> wrote:
>>> In the latest release of "Large Scale C++", Lakos describes a wrapping
>>> technique that involves passing pointers between objects of unrelated
>>> classes (say, class A and class B), where the pointer is of a third
>>> type (say, class T) that is only declared, but does not need to be
>>> ever defined. The pointer value is internally reinterpret_casted to
>>> whatever is needed, but this detail is not visible to the user of
>>> these classes. So:
>>>
>>> class T; // declared, but never defined
>>>
>>> class A { public: T * getT(); // reinterpret_cast<T *>(...) };
>>>
>>> class B { public: void foo(T * p); // reinterpret_cast<something *>(p)
>>> };
>>>
>>> Without this idiom, the pointer would be passed around in the form of
>>> void*, but the use of class T adds some type safety, enables
>>> overloading, etc.
>>
>> What type safety? It is worse as you don't need reinterpret cast
>> in case of passing void*....
>
> Double static_cast to void* and back to T* is equivalent to a
> reinterpret_cast.

Yes, but what's the point of void* then? We used char* before void,
and now when void* is introduced, we should use it.

>
> Actually, with the "T*" interfaces, one does not need any
> reinterpret_casts in the C++ implementation code which sees the class
> definitions. Upcasting in a class hierarchy is implicit and downcasting
> can be done by a single static_cast.
>
> reinterpret_cast would be needed in the client code, which does not see
> class definitions. However, reinterpret_cast is very under-specified and
> may easily fail (especially in case of multiple or virtual inheritance,
> but I'm pretty sure the results are formally undefined or
> implementation-specific also in simpler scenarios).
>
> So I would ban such casts in the client side code and provide explicit
> conversion functions instead if the client code really needs to cast
> between different derived class pointer types. In short,
> reinterpret_cast is not a proper way to implement anything, I'm a bit
> surprised if Lakos really recommends it.

+1

[Paavo Helde]

This idiom is often used in library interfaces, mostly in C, but also in
C++. For example, the main "handle" type SSL in openssl 1.1 is defined as:

typedef struct ssl_st SSL;

without definition of ssl_st. The public functions are defined in terms
of SSL*. This is in contrast to openssl 1.0 where the struct definitions
were public; the change was specifically done to hide implementation
details - see "https://wiki.openssl.org/index.php/OpenSSL_1.1.0_Changes".

Another example are Python C bindings (CPython), where most of the
interfaces are defined in terms of "base class" pointer PyObject*, but
in reality there is no PyObject object, all objects are of some
"derived" class like PyLongObject, etc. So the implementation code
contains a lot of casting which are effectively reinterpret_cast-s
because there is no inheritance in C and all those structs are
technically unrelated.

In C++ the dreadful reinterpret_cast is actually not needed if things
are done properly, static_cast is guaranteed to do the correct thing in
derived class hierarchies.

Of course, using such raw pointers more suits a C-style interface. BTW,
it is a great idea to provide a C-style interface for a C++ library,
this allows for its much wider use and avoids some technical
complications related to the lack of standardized ABI in C++.

However, in pure C++ one typically expects to have automatic resource
release in some destructor where needed, so the raw pointer should be
packed into some small "handle" class, either std::shared_ptr or
something more elaborate. In this way we arrive to the pimpl idiom,
which is indeed often used in C++ for hiding implementation details.

[Melzzzzz]

On 2020-03-23, Paavo Helde <myfir...@osa.pri.ee> wrote:

Actually in Haskell this idiom is necessary as then one introduce,
type check in compiler.
eg:
newtype CB3 = CB3 (Ptr CB3)
type Callback3 = (PtrCallback -> PtrCallback1 -> PtrCallback ->
PtrCallback1 -> PtrCallback ->
PtrCallback2 -> CB3 -> PtrCallback4 -> IO ())
type PtrCallback3 = FunPtr Callback3

No casts needed just to associate type to opaque data.

[Maciej Sobczak]

> >> class T; // declared, but never defined
> >>
> >> class A { public: T * getT(); // reinterpret_cast<T *>(...) };
> >>
> >> class B { public: void foo(T * p); // reinterpret_cast<something *>(p)
> >> };

> > What type safety?

Without client-side casts, you can only call B::foo with what you got from A::getT. If this code pattern was based on void*, then you might be able to call B::foo with whatever and since conversion to void* is implicit, there would be no hint that there is anything wrong with the code. In particular, no help from the compiler.

So, using T* binds the producer and consumer in a way that only a proper use compiles without errors:

my_b.foo(my_a.getT()); // OK

my_b.foo(my_c.getU()); // error

> Actually, with the "T*" interfaces, one does not need any
> reinterpret_casts in the C++ implementation code which sees the class
> definitions. Upcasting in a class hierarchy is implicit and downcasting
> can be done by a single static_cast.

But there is no hierarchy here and the value of this idiom is that there is no need for any hierarchy.

> reinterpret_cast would be needed in the client code

No, see above.
Actually, it is the presence of reinterpret_cast that might indicate a potential problem, or at least a usage that deserves attention in code reviews.

> So I would ban such casts in the client side code

Again: there are no casts in the client side code. That's the whole point. The cast is an implementation detail of the classes/functions that participate in this idiom.

> I'm a bit
> surprised if Lakos really recommends it.

He does not *recommend* it in the sense of endorsement, but presents it as an idiom that has emerged and is in use. The other responses about Win and Python APIs show that this idiom is not even a recent one and can be found in pure C, too.

[Paavo Helde]

On 23.03.2020 13:13, Maciej Sobczak wrote:
>>>> class T; // declared, but never defined
>>>>
>>>> class A { public: T * getT(); // reinterpret_cast<T *>(...) };
>>>>
>>>> class B { public: void foo(T * p); // reinterpret_cast<something *>(p)
>>>> };
>
>>> What type safety?
>
> Without client-side casts, you can only call B::foo with what you got from A::getT. If this code pattern was based on void*, then you might be able to call B::foo with whatever and since conversion to void* is implicit, there would be no hint that there is anything wrong with the code. In particular, no help from the compiler.
>
> So, using T* binds the producer and consumer in a way that only a proper use compiles without errors:
>
> my_b.foo(my_a.getT()); // OK
>
> my_b.foo(my_c.getU()); // error
>
>> Actually, with the "T*" interfaces, one does not need any
>> reinterpret_casts in the C++ implementation code which sees the class
>> definitions. Upcasting in a class hierarchy is implicit and downcasting
>> can be done by a single static_cast.
>
> But there is no hierarchy here and the value of this idiom is that there is no need for any hierarchy.

Ah, I see I have misunderstood the concept and spoke about something
else. Somehow I assumed that A and B are themselves hidden
implementation classes, and I assumed T is finally defined somewhere.

Now it seems there is actually some unrelated hidden class C, and a C*
pointer is reinterpret_cast into T*, and back to C* later. But what's
the point? Why shouldn't C be just forward-declared in the header with
'class C'? What's the benefit of having a non-existent class T?

As Alf said, Win32 interfaces are a bit similar. For example, FILE is
defined as

typedef struct _iobuf
{
void* _Placeholder;
} FILE;

It looks like this is not the actual implementation, meaning that any
FILE* pointer is first cast to something else in the implementation
code. I have always assumed this is just another legacy madness from
Microsoft which was maybe necessary in 16-bit DOS, or something.

I do not recall such usage in genuine C++ code.

[Maciej Sobczak]

> Now it seems there is actually some unrelated hidden class C, and a C*
> pointer is reinterpret_cast into T*, and back to C* later.

Yes, in the pattern described in the book, there was indeed type C involved and T was used to hide it.

> But what's
> the point? Why shouldn't C be just forward-declared in the header with
> 'class C'? What's the benefit of having a non-existent class T?

In that case (from the book) the whole idea was that A and B were wrappers for something (say X and Y) that was not to be exposed on the public view. Indeed, the third class that you suspect to exist, was a genuine base class of one of these wrapped/hidden constructs. The idea was to wrap that hierarchy without the need to repeat it at the wrapper level, because the hierarchical nature of the wrapped stuff could have been an implementation detail.
So there was:

class Base { ... };
class X : public Base { ... }; // hierarchy
class Y { void foo(Base &); };

and this stuff was wrapped as:

class T;
class A { T* getT(); }; // no hierarchy
class B { void foo(T*); };

As you guessed, T* is only the token that carries that actual Base*.
But the hierarchical nature of the wrapped stuff does not leak to the wrapper level, so can be changed to something else without disturbing the wrapper interface.

It makes sense to me and I find it to be a sufficient justification for some dirty tricks under the hood (like reinterpret_casts Base* <-> T* ), and I was interested whether this idea has found some other uses.

> As Alf said, Win32 interfaces are a bit similar. For example, FILE is
> defined as
>
> typedef struct _iobuf
> {
> void* _Placeholder;
> } FILE;
>
> It looks like this is not the actual implementation,

I think the idea was similar and the only reason for the whole dummy struct was that there is no way to typedef the incomplete type and declaration alone would not allow to use "FILE" without the "struct" keyword.

In any case, resource handles seem to be good candidates for this idiom.