offsetof

[Paavo Helde]

How to make offsetof() usage legal in this example? Is there a better
way instead of using offsetof? Basically I want to restore a full object
pointer from a pointer to a member object. This example comes from some
heavily space-optimized code, so adding any new helper data members
anywhere is out of question.

The code probably works fine with all compilers/platforms it will ever
see, but the compiler spits out nasty warnings, because there is some
data both in the base and derived classes, making the class not
"standard layout".

#include <cassert>
#include <cstddef>

class Base {
int x;
public:
Base(): x(0) {}
};

class Inner {
int y;
public:
Inner(): y(0) {}
};

class Derived : public Base {
Inner inner;
public:
Inner* GetInner() {
return &inner;
}
static Derived* OwnerOf(Inner* inner);
};

Derived* Derived::OwnerOf(Inner* inner) {
return reinterpret_cast<Derived*>(
reinterpret_cast<char*>(inner) - offsetof(Derived, inner));
}

int main() {
Derived a;
Inner* p = a.GetInner();
Derived* q = Derived::OwnerOf(p);
assert(q == &a);
}

> g++ -Wall test1.cpp -std=c++17
In file included from /usr/include/sys/_types.h:159:0,
from /usr/include/sys/reent.h:15,
from /usr/include/stdlib.h:18,
from
/usr/lib/gcc/x86_64-pc-cygwin/7.3.0/include/c++/cstdlib:75,
from test1.cpp:3:
test1.cpp: In static member function 'static Derived*
Derived::OwnerOf(Inner*)':
test1.cpp:27:79: warning: offsetof within non-standard-layout type
'Derived' is undefined [-Winvalid-offsetof]
return reinterpret_cast<Derived*>( reinterpret_cast<char*>(inner) -
offsetof(Derived, inner));

[Marcel Mueller]

Am 24.05.19 um 13:31 schrieb Paavo Helde:

> How to make offsetof() usage legal in this example?

I think there is no legal variant for your use case. AFAIK offsetof and
inheritance shouldn't be used for the same type.

> Is there a better way instead of using offsetof?

Well, normally you should use (multiple) inheritance in this case. I.e.

class Derived : public Base, private Inner
{ public:
Inner* GetInner() { return this; }
static Derived* OwnerOf(Inner* inner) { return (Derived*)inner; }
};

Now it is just a downcast.

Assignment to Inner (if required) could be done by slicing.


Marcel

[Paavo Helde]

On 24.05.2019 18:54, Marcel Mueller wrote:
> Am 24.05.19 um 13:31 schrieb Paavo Helde:
>> How to make offsetof() usage legal in this example?
>
> I think there is no legal variant for your use case. AFAIK offsetof and
> inheritance shouldn't be used for the same type.
>
>> Is there a better way instead of using offsetof?
>
> Well, normally you should use (multiple) inheritance in this case. I.e.
>
> class Derived : public Base, private Inner
> { public:
> Inner* GetInner() { return this; }
> static Derived* OwnerOf(Inner* inner) { return (Derived*)inner; }
> };
>
> Now it is just a downcast.

Thanks! Apparently this works, but I'm pretty surprised it does. The
OwnerOf() function does not have any knowledge that the passed-in inner
pointer actually belongs to an Inner base class object inside a Derived
object, but it still agrees to perform the cast. Amazing, usually C++ is
much more picky!

Well, I guess I can learn something new every day!

[Marcel Mueller]

Am 24.05.19 um 19:21 schrieb Paavo Helde:

> On 24.05.2019 18:54, Marcel Mueller wrote:
>> class Derived : public Base, private Inner
>> { public:
>>      Inner* GetInner() { return this; }
>>      static Derived* OwnerOf(Inner* inner) { return (Derived*)inner; }
>> };
>>
>> Now it is just a downcast.
>
> Thanks! Apparently this works, but I'm pretty surprised it does. The
> OwnerOf() function does not have any knowledge that the passed-in inner
> pointer actually belongs to an Inner base class object inside a Derived
> object, but it still agrees to perform the cast.

Well, if you pass something else you get UB.

Only if you have polymorphic types you could use dynamic_cast<> which
has defined behavior in such cases.

> Amazing, usually C++ is much more picky!

I can't confirm this. You can do almost any dirty hack in C++. Container
classes with type erasure, smart pointers that are /binary/ compatible
to some C pointers like const char*, efficient arrays with members like
_size in a single allocation with the array content and without an
offset from the objects pointer to the start of the array, type safe and
strongly thread safe smart pointers without DWCAS and so on. Whether
this is defined behavior according to the language standard is another
question. Sometimes it is only defined behavior on some platforms.


Marcel

[Paavo Helde]

On 24.05.2019 21:23, Marcel Mueller wrote:
> Am 24.05.19 um 19:21 schrieb Paavo Helde:
>> On 24.05.2019 18:54, Marcel Mueller wrote:
>>> class Derived : public Base, private Inner
>>> { public:
>>> Inner* GetInner() { return this; }
>>> static Derived* OwnerOf(Inner* inner) { return (Derived*)inner; }
>>> };
>>>
>>> Now it is just a downcast.
>>
>> Thanks! Apparently this works, but I'm pretty surprised it does. The
>> OwnerOf() function does not have any knowledge that the passed-in
>> inner pointer actually belongs to an Inner base class object inside a
>> Derived object, but it still agrees to perform the cast.
>
> Well, if you pass something else you get UB.
>
> Only if you have polymorphic types you could use dynamic_cast<> which
> has defined behavior in such cases.
>
>> Amazing, usually C++ is much more picky!
>
> I can't confirm this. You can do almost any dirty hack in C++.

Sure, but I was assuming offsetof is one of such "dirty hacks", not a
single static_cast ;-)

[Öö Tiib]

I copy-paste some legaleze from random draft of standard
for the mood:

About the offsetof in [support.types.layout]:
| The macro offsetof(type, member-designator) has the same semantics
| as the corresponding macro in the C standard library header <stddef.h>,
| but accepts a restricted set of type arguments in this International
| Standard.
| Use of the offsetof macro with a type other than a standard-layout class
| is conditionally-supported. [Note that offsetof is required to work as
| specified even if unary operator& is overloaded for any of the types
| involved.]
| The expression offsetof(type, member-designator ) is never type-
| dependent and it is value-dependent if and only if type is dependent.
| The result of applying the offsetof macro to a static data member or a
| function member is undefined.
| No operation invoked by the offsetof macro shall throw an exception and
| noexcept(offsetof(type, member-designator )) shall be true.

The "conditionally-supported" is defined [defns.cond.supp] as:
| program construct that an implementation is not required to support
| [ Note: Each implementation documents all conditionally-supported
| constructs that it does not support. — end note ]

So implementation has right to define for what kind of types (if any)
besides standard-layout classes it does not support offsetof. And
again any mechanisms for a programmer to detect it compile-time
are undescribed.
That makes offsetof implementation-defined and non-portable for
anything but standard-layout classes.

[Richard Damon]

One thing to watch out here, is that the original question commented
about space constraints and wanting to avoid helper pointers, but this
multiple inheritance situation may implicitly create such pointers (at
least if you want to get the right answer)

Also, using (Derived*)inner in this sense is using a C style cast, which
will in this case be a reinterpret_cast if I am parsing things right,
which will only work if inner is at offsetof of 0. The side effect of
this is that the reinterpret_cast is UB if the inner isn't part of
Derived or not at offset 0 (and since Base has data members, it really
can't have an offset of 0). You could use instead a dynamic_cast which
will detect the issue and return a 0. To use the dynamic_cast, inner
will need to have a virtual function so it can have RTTI (which gets us
back to the need for helpers).

In C++, using C Style casts is generally a bad idea, because you have
lost the ability to detect that you are doing something dangerous. A C
style cast can convert ANY pointer to ANY OTHER pointer, regardless of
if it really makes sense. Use a static_cast if you can, as that is much
safer, and it will reject code trying to do things it shouldn't be
doing. dynamic_cast is also safe, but more expensive, as it will need to
inspect the RTTI info of the object. reinterpret_cast needs to be used
with care, as does const_cast, and these being explicit are easier to
review.

Going back to the original question, the fact that you got a warning,
and not an error, says that the implementation may support this
operation, even though the standard doesn't require it to. Thus we get
back to the initial question, to do this by the standard, inner will
need to increase by the size of a pointer, either to add the implicit
vtable pointer to get RTTI, or adding a pointer of our own to provide
the owner. Or, if the implementation supports the extension, use
offsetof on a non-standard-layout structure. This case SHOULD have a
well defined layout for most systems, it is just that the standard gives
the implementation some flexability. offsetof probably could have been
defined for a wider range of structures, but standard-layout was a
defined class of them that it could clearly work and handles the need
for compatibility with C.

[Paavo Helde]

On 27.05.2019 18:31, Richard Damon wrote:
> On 5/24/19 11:54 AM, Marcel Mueller wrote:
>> Am 24.05.19 um 13:31 schrieb Paavo Helde:
>>> How to make offsetof() usage legal in this example?
>>
>> I think there is no legal variant for your use case. AFAIK offsetof and
>> inheritance shouldn't be used for the same type.
>>
>>> Is there a better way instead of using offsetof?
>>
>> Well, normally you should use (multiple) inheritance in this case. I.e.
>>
>> class Derived : public Base, private Inner
>> { public:
>> Inner* GetInner() { return this; }
>> static Derived* OwnerOf(Inner* inner) { return (Derived*)inner; }
>> };
>>
>> Now it is just a downcast.
>>
>> Assignment to Inner (if required) could be done by slicing.
>>
>>
>> Marcel
>
> One thing to watch out here, is that the original question commented
> about space constraints and wanting to avoid helper pointers, but this
> multiple inheritance situation may implicitly create such pointers (at
> least if you want to get the right answer)
>
> Also, using (Derived*)inner in this sense is using a C style cast, which
> will in this case be a reinterpret_cast if I am parsing things right,

No, it's a static_cast.

> which will only work if inner is at offsetof of 0.

No, static_cast works correctly here and adjusts the pointer as needed.
That's the piece which surprised me.

> The side effect of
> this is that the reinterpret_cast is UB if the inner isn't part of
> Derived or not at offset 0 (and since Base has data members, it really
> can't have an offset of 0).

reinterpret_cast would work incorrectly indeed.

> You could use instead a dynamic_cast which
> will detect the issue and return a 0. To use the dynamic_cast, inner
> will need to have a virtual function so it can have RTTI (which gets us
> back to the need for helpers).

In my case there cannot be virtuals, every byte is counted.

>
> In C++, using C Style casts is generally a bad idea, because you have
> lost the ability to detect that you are doing something dangerous. A C
> style cast can convert ANY pointer to ANY OTHER pointer, regardless of
> if it really makes sense. Use a static_cast if you can, as that is much
> safer, and it will reject code trying to do things it shouldn't be
> doing. dynamic_cast is also safe, but more expensive, as it will need to
> inspect the RTTI info of the object. reinterpret_cast needs to be used
> with care, as does const_cast, and these being explicit are easier to
> review.

Agreed. I used static_cast in my code.

> Going back to the original question, the fact that you got a warning,
> and not an error, says that the implementation may support this
> operation, even though the standard doesn't require it to. Thus we get
> back to the initial question, to do this by the standard, inner will
> need to increase by the size of a pointer, either to add the implicit
> vtable pointer to get RTTI, or adding a pointer of our own to provide
> the owner. Or, if the implementation supports the extension, use
> offsetof on a non-standard-layout structure. This case SHOULD have a
> well defined layout for most systems, it is just that the standard gives
> the implementation some flexability. offsetof probably could have been
> defined for a wider range of structures, but standard-layout was a
> defined class of them that it could clearly work and handles the need
> for compatibility with C.

For me static_cast worked perfectly here, no need for offsetof.

[Richard Damon]

On 5/27/19 12:47 PM, Paavo Helde wrote:
> On 27.05.2019 18:31, Richard Damon wrote:
>> On 5/24/19 11:54 AM, Marcel Mueller wrote:
>>> Am 24.05.19 um 13:31 schrieb Paavo Helde:
>>>> How to make offsetof() usage legal in this example?
>>>
>>> I think there is no legal variant for your use case. AFAIK offsetof and
>>> inheritance shouldn't be used for the same type.
>>>
>>>> Is there a better way instead of using offsetof?
>>>
>>> Well, normally you should use (multiple) inheritance in this case. I.e.
>>>
>>> class Derived : public Base, private Inner
>>> { public:
>>>      Inner* GetInner() { return this; }
>>>      static Derived* OwnerOf(Inner* inner) { return (Derived*)inner; }
>>> };
>>>
>>> Now it is just a downcast.
>>>
>>> Assignment to Inner (if required) could be done by slicing.
>>>
>>>
>>> Marcel
>>
>> One thing to watch out here, is that the original question commented
>> about space constraints and wanting to avoid helper pointers, but this
>> multiple inheritance situation may implicitly create such pointers (at
>> least if you want to get the right answer)
>>
>> Also, using (Derived*)inner in this sense is using a C style cast, which
>> will in this case be a reinterpret_cast if I am parsing things right,
>
> No, it's a static_cast.

If you can use a static_cast the it does work, and thinking harder, yes,
casts from base to derived can be done with static_cast which give
usable values as long as the object IS an object derived from the base.

>
>> which will only work if inner is at offsetof of 0.
>
> No, static_cast works correctly here and adjusts the pointer as needed.
> That's the piece which surprised me.

Yes, if it becomes a static cast, then of offset is applied (one BIG
reason to avoid the C cast, it isn't always clear what you are getting)

>
>> The side effect of
>> this is that the reinterpret_cast is UB if the inner isn't part of
>> Derived or not at offset 0 (and since Base has data members, it really
>> can't have an offset of 0).
>
> reinterpret_cast would work incorrectly indeed.
>
>> You could use instead a dynamic_cast which
>> will detect the issue and return a 0. To use the dynamic_cast, inner
>> will need to have a virtual function so it can have RTTI (which gets us
>> back to the need for helpers).
>
> In my case there cannot be virtuals, every byte is counted.

Yes, I presumed so.

>
>>
>> In C++, using C Style casts is generally a bad idea, because you have
>> lost the ability to detect that you are doing something dangerous. A C
>> style cast can convert ANY pointer to ANY OTHER pointer, regardless of
>> if it really makes sense. Use a static_cast if you can, as that is much
>> safer, and it will reject code trying to do things it shouldn't be
>> doing. dynamic_cast is also safe, but more expensive, as it will need to
>> inspect the RTTI info of the object. reinterpret_cast needs to be used
>> with care, as does const_cast, and these being explicit are easier to
>> review.
>
> Agreed. I used static_cast in my code.

Good!

[Marcel Mueller]

Am 27.05.19 um 20:03 schrieb Richard Damon:

>> No, static_cast works correctly here and adjusts the pointer as needed.
>> That's the piece which surprised me.
>
> Yes, if it becomes a static cast, then of offset is applied (one BIG
> reason to avoid the C cast, it isn't always clear what you are getting)

C style casts will never be a dynamic_cast, i.e. they never use RTTI.


Marcel

[Chris Vine]

Out of interest, do you know what cast can only be achieved with a
defined result in C++ using a C-style cast (that is, it cannot be
achieved with static_cast, reinterpret_cast, const_cast or
dynamic_cast, or any combination thereof, legally)?

[Alf P. Steinbach]

I don't think that's a good way to judge someone's knowledge of the
language.

Anyway it's a cast to inaccessible base.


Cheers!,

- Alf

[Chris Vine]

It is, and it was for general interest and amusement as a corollary
(as the thing that only C-style casts can do) of the case Marcel
mentioned (the thing that C-style casts cannot do).

[Richard Damon]

I would expect it also work the other way to, from inaccessible base to
derived (but would need to double check that before using it that way).

[Chris Vine]

It does for non-virtual base classes, according to §5.4/4 of C++14:

The same semantic restrictions and behaviors apply [to a C-style
cast], with the exception that in performing a static_cast in the
following situations the conversion is valid even if the base class is
inaccessible:

[... conversion from derived to unambiguous base ...]

[... conversion from pointer to member of derived to pointer to
member of non-virtual base ...]

a pointer to an object of an unambiguous non-virtual base class type,
a glvalue of an unambiguous non-virtual base class type, or a pointer
to member of an unambiguous non-virtual base class type may be
explicitly converted to a pointer, a reference, or a pointer to member
of a derived class type, respectively

[Tim Rentsch]

> [...]

How about emulating offsetof() using a static member function?
(Disclaimer: please excuse my writing C style casts instead of
C++ style casts.)

class Derived : public Base {
Inner inner;
public:
Inner* GetInner() {
return &inner;
}

static Derived* OwnerOf(Inner* inner){
alignas( Derived ) unsigned char it[ sizeof (Derived) ];
unsigned delta = Derived::offsetof_inner( (Derived*) it );
return (Derived*)( (char*)inner - delta );
}
private:
static unsigned
offsetof_inner( Derived *d ){
return (char*)&d->inner_ - (char*)d;
}
};

I like this approach better than using a private base class.
AFAICT it has well-defined behavior (no accesses, and no
non-static member functions, with the "synthesized" pointer to a
Derived). No diagnostics from g++ or clang. Produces good code
with g++ at level -O1.