Rationale, FAQ for P0194R0 (Static reflection)

[Matus Chochlik]

Dear list,

Some time ago we've decided that we will split the 'Static reflection' paper into two.

One which will be terse and strictly to the point and will eventually result in the wording to be included in the standard, and the second which will cover the rationale, design decisions, use cases and examples and the answers to the frequently asked questions.

The current draft of the second paper can be found here:

http://kifri.fri.uniza.sk/~chochlik/reflexpr/DxxxxR0.pdf

It is not finished but any comments, additional questions for the FAQ section or other feedback is very welcome.

You very probably don't need to read the whole thing (you of course can if you will). Instead browse through the contents and look up the things you are interested in.

Best regards,

Matus

[Klaim - Joël Lamotte]

Hi,

I read it all so here are my comments:

1. The whole feature and approach seems good to me (taking into account the different projects and companies I work for)

   except maybe the verbosity which might not be as easily fixed as you suggest with library functions.

   I think it will become a maintenance issue in the future, but for now I don't have a suggestion to improve it except minor ones like:

  1.1. maybe just use template objects and have meta::get_name<mtint>  instead of meta::get_name_v<mint>

  1.2. maybe remove the "get" prefix to have meta::name<mtint>? I find it better to go without get in this specific case because the namespace greatly helps clarify the intent of the operation.

2. To me most of the logging examples are not completely convincing, they feel like macros will still be necessary to hide details.

What would be useful would be a compile-time version of source_location which would make a template logging function

knows the meta-context in which it was called. It would be more powerful than the runtime equivalent as it would be able

to automatically generate the logging, find the data to log etc. (but would work only with template functions)

Something that would result in:

int foo( Bar bar, K k)

{

    log_called();  // implicitely log the function name, source code location and arguments names, types and values.

    // work...

}

To do that there seems be a need for a way to capture "access" to the local variables at runtime, to be able to log their values.

Maybe this could be part of the possible extensions section in the document.

3. Another possible extension which I think might be difficult but very very useful too would be a way to be able to

inject a call after or before another any call of a function.

This would enable, for example, the "all member functions will log" case by doing this:

class A

{

   using LogMembersCalls = ...reflexpr(this::class)...;  // inject logging in all member functions, before and after their internal code

   void foo(){  /* ... work ... * } // no need to forget adding the log

   void bar(){  /* ... work ... * } // no need to forget adding the log

};

It would also help with micro-profiling libraries and other cases.

4. Here are the use cases I have in the different projects I work for:

 4.1. Structure Of Array: it is not mentioned in the paper.

     Indeed I don't see a way to use the current reflection proposal to do SOA without breaking the compatibility

     with functions which are unaware about SOA arguments. Maybe the operator-dot proposal helps with this

     by providing a way to manipulate proxies to SOA data, but it's still surprising that it is not mentioned

     in the paper.

     Have their been actual experimentation about implementing SOA containers?

 4.2. Remote proxy: we use both a custom reflection system and a object/entity proxy object system to

    manipulate a distant object interface. Without something like the operator dot proposal, the current reflection

    facility only provide approximately half of what we currently implement (in a very discutable and hard to maintain way).

    I expect some potential extensions to help with this.

5. If there is an implementation, I think a link to it in the beginning of the paper would be useful.

6. Globally readable to me, no complain on this point (except the redundant code parts but it's necessary to understand the examples).

--
You received this message because you are subscribed to the Google Groups "SG 7 - Reflection" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reflection+...@isocpp.org.
To post to this group, send email to refle...@isocpp.org.
Visit this group at https://groups.google.com/a/isocpp.org/group/reflection/.
For more options, visit https://groups.google.com/a/isocpp.org/d/optout.

[Matus Chochlik]

Hi,

On Fri, Apr 22, 2016 at 4:53 PM, Klaim - Joël Lamotte <mjk...@gmail.com> wrote:

Hi,

I read it all so here are my comments:

1. The whole feature and approach seems good to me (taking into account the different projects and companies I work for)

   except maybe the verbosity which might not be as easily fixed as you suggest with library functions.

   I think it will become a maintenance issue in the future, but for now I don't have a suggestion to improve it except minor ones like:

Fair enough, using reflection won't be always trivial, but I think that we can do a decent job with the library in many cases.

 

  1.1. maybe just use template objects and have meta::get_name<mtint>  instead of meta::get_name_v<mint>

Here we were trying to be consistent with the type traits where you also have `std::is_same` and `std::is_same_v`. The libraries on top of the basic reflection do not have to stick to this convention.

 

  1.2. maybe remove the "get" prefix to have meta::name<mtint>? I find it better to go without get in this specific case because the namespace greatly helps clarify the intent of the operation.

There are two reasons why the `get_` prefix was added. Again consistency with the rest of the std library and more importantly, to avoid conflicts with C++ keywords in the future.

Once the reflection of templates is added, we will have a Meta-Instantiation (reflecting template instantiations) and this will have an operation returning the reflection of the template,

which we plan to call `get_template`, if the `get_` is stripped then the name will be invalid and we'll have to come up with a different one.
And we are expecting that there will be several instances of this.

 

2. To me most of the logging examples are not completely convincing, they feel like macros will still be necessary to hide details.

What would be useful would be a compile-time version of source_location which would make a template logging function

knows the meta-context in which it was called. It would be more powerful than the runtime equivalent as it would be able

to automatically generate the logging, find the data to log etc. (but would work only with template functions)

Something that would result in:

Yes, it is true that for it not to be very verbose it would be necessary at the moment to wrap the repetitive; reflexpr(this::function) & co. into a macro.

I was thinking about how to overcome this, and all solutions that I've come-up with so far would require something like a Meta-FunctionInvocation which would have access to the whole call stack.

 

int foo( Bar bar, K k)

{

    log_called();  // implicitely log the function name, source code location and arguments names, types and values.

    // work...

}

To do that there seems be a need for a way to capture "access" to the local variables at runtime, to be able to log their values.

Maybe this could be part of the possible extensions section in the document.

Something like this is discussed in N4451 appendix A.19.3 (p.59) as a part of Meta-Parameter where you can get the pointer to function parameters.
We'll gradually add all/most of the stuff pruned between N4451 and P0194R0 back to the rationale paper so that it's there to be discussed.

 

3. Another possible extension which I think might be difficult but very very useful too would be a way to be able to

inject a call after or before another any call of a function.

This would enable, for example, the "all member functions will log" case by doing this:

Here we go into AOP territory, which is a whole different ball game, but I agree that it's related to reflection and would be nice to have.

 

class A

{

   using LogMembersCalls = ...reflexpr(this::class)...;  // inject logging in all member functions, before and after their internal code

   void foo(){  /* ... work ... * } // no need to forget adding the log

   void bar(){  /* ... work ... * } // no need to forget adding the log

};

It would also help with micro-profiling libraries and other cases.

Appreciated.

4. Here are the use cases I have in the different projects I work for:

 4.1. Structure Of Array: it is not mentioned in the paper.

     Indeed I don't see a way to use the current reflection proposal to do SOA without breaking the compatibility

     with functions which are unaware about SOA arguments. Maybe the operator-dot proposal helps with this

     by providing a way to manipulate proxies to SOA data, but it's still surprising that it is not mentioned

     in the paper.

     Have their been actual experimentation about implementing SOA containers?

It is now. Section 4.9 discusses this use case.

 

 4.2. Remote proxy: we use both a custom reflection system and a object/entity proxy object system to

    manipulate a distant object interface. Without something like the operator dot proposal, the current reflection

    facility only provide approximately half of what we currently implement (in a very discutable and hard to maintain way).

    I expect some potential extensions to help with this.

This is related to the SoA case, and the main obstacle here is IMHO that we don't have the ability to "construct" identifiers without the help of the preprocessor.

 

5. If there is an implementation, I think a link to it in the beginning of the paper would be useful.

OK, noted.

 

6. Globally readable to me, no complain on this point (except the redundant code parts but it's necessary to understand the examples).

Thanks for the feedback! If you have any more, it's always welcome.

Matus

[Klaim - Joël Lamotte]

On 22 April 2016 at 17:29, Matus Chochlik <choc...@gmail.com> wrote:

Hi,

On Fri, Apr 22, 2016 at 4:53 PM, Klaim - Joël Lamotte <mjk...@gmail.com> wrote:

Hi,

I read it all so here are my comments:

1. The whole feature and approach seems good to me (taking into account the different projects and companies I work for)

   except maybe the verbosity which might not be as easily fixed as you suggest with library functions.

   I think it will become a maintenance issue in the future, but for now I don't have a suggestion to improve it except minor ones like:

Fair enough, using reflection won't be always trivial, but I think that we can do a decent job with the library in many cases.

 

  1.1. maybe just use template objects and have meta::get_name<mtint>  instead of meta::get_name_v<mint>

Here we were trying to be consistent with the type traits where you also have `std::is_same` and `std::is_same_v`. The libraries on top of the basic reflection do not have to stick to this convention.

 

  1.2. maybe remove the "get" prefix to have meta::name<mtint>? I find it better to go without get in this specific case because the namespace greatly helps clarify the intent of the operation.

There are two reasons why the `get_` prefix was added. Again consistency with the rest of the std library and more importantly, to avoid conflicts with C++ keywords in the future.

Once the reflection of templates is added, we will have a Meta-Instantiation (reflecting template instantiations) and this will have an operation returning the reflection of the template,

which we plan to call `get_template`, if the `get_` is stripped then the name will be invalid and we'll have to come up with a different one.
And we are expecting that there will be several instances of this.

Fair enough.

 

 

2. To me most of the logging examples are not completely convincing, they feel like macros will still be necessary to hide details.

What would be useful would be a compile-time version of source_location which would make a template logging function

knows the meta-context in which it was called. It would be more powerful than the runtime equivalent as it would be able

to automatically generate the logging, find the data to log etc. (but would work only with template functions)

Something that would result in:

Yes, it is true that for it not to be very verbose it would be necessary at the moment to wrap the repetitive; reflexpr(this::function) & co. into a macro.

I was thinking about how to overcome this, and all solutions that I've come-up with so far would require something like a Meta-FunctionInvocation which would have access to the whole call stack.

 

Why not just the caller's stack frame? I think it might be enough.

 

int foo( Bar bar, K k)

{

    log_called();  // implicitely log the function name, source code location and arguments names, types and values.

    // work...

}

To do that there seems be a need for a way to capture "access" to the local variables at runtime, to be able to log their values.

Maybe this could be part of the possible extensions section in the document.

Something like this is discussed in N4451 appendix A.19.3 (p.59) as a part of Meta-Parameter where you can get the pointer to function parameters.
We'll gradually add all/most of the stuff pruned between N4451 and P0194R0 back to the rationale paper so that it's there to be discussed.

 

By the way, my understanding is that there will be a TS for reflection? Is it already ongoing?

 

3. Another possible extension which I think might be difficult but very very useful too would be a way to be able to

inject a call after or before another any call of a function.

This would enable, for example, the "all member functions will log" case by doing this:

Here we go into AOP territory, which is a whole different ball game, but I agree that it's related to reflection and would be nice to have.

 

Just in case this pov helps: I tend to see this idea like custom special functions, like destructors, because they are also

injected automatically in user's code. Maybe arguing that destructors is more or less a rigid precedent of this potential might help make a point.

4. Here are the use cases I have in the different projects I work for:

 4.1. Structure Of Array: it is not mentioned in the paper.

     Indeed I don't see a way to use the current reflection proposal to do SOA without breaking the compatibility

     with functions which are unaware about SOA arguments. Maybe the operator-dot proposal helps with this

     by providing a way to manipulate proxies to SOA data, but it's still surprising that it is not mentioned

     in the paper.

     Have their been actual experimentation about implementing SOA containers?

It is now. Section 4.9 discusses this use case.

Good, I just read it. More below.

 

 

 4.2. Remote proxy: we use both a custom reflection system and a object/entity proxy object system to

    manipulate a distant object interface. Without something like the operator dot proposal, the current reflection

    facility only provide approximately half of what we currently implement (in a very discutable and hard to maintain way).

    I expect some potential extensions to help with this.

This is related to the SoA case, and the main obstacle here is IMHO that we don't have the ability to "construct" identifiers without the help of the preprocessor.

 

I now remember reading about the identifier operator.

About 4.9 and 5.4.1, I'm not totally sure if there is a need for always generating identifier by passing through a string, even if the feature would help in other cases.

Maybe there could be a way to manipulate identifiers not a string but as something that compile-time-only that can take strings as concatenation and generate

an identifier? That way you would not need to have compile-time string facilities, only compiler "magic" (but that might be too much magic).

I'm not sure how to make an example of this for now, I'll think about it.

 

5. If there is an implementation, I think a link to it in the beginning of the paper would be useful.

OK, noted.

 

6. Globally readable to me, no complain on this point (except the redundant code parts but it's necessary to understand the examples).

Thanks for the feedback! If you have any more, it's always welcome.

Matus

[Matus Chochlik]

On Fri, Apr 22, 2016 at 8:44 PM, Klaim - Joël Lamotte <mjk...@gmail.com> wrote:

[8<]

Yes, it is true that for it not to be very verbose it would be necessary at the moment to wrap the repetitive; reflexpr(this::function) & co. into a macro.

I was thinking about how to overcome this, and all solutions that I've come-up with so far would require something like a Meta-FunctionInvocation which would have access to the whole call stack.

 

Why not just the caller's stack frame? I think it might be enough.

I meant being able to *incrementally/gradually* access the whole call stack, something analogous to a Meta-Scoped reflecting a deeply nested declaration being able to get back to the Meta-GlobalScope via a series of invocations of `get_scope`.

 

 

int foo( Bar bar, K k)

{

    log_called();  // implicitely log the function name, source code location and arguments names, types and values.

    // work...

}

To do that there seems be a need for a way to capture "access" to the local variables at runtime, to be able to log their values.

Maybe this could be part of the possible extensions section in the document.

Something like this is discussed in N4451 appendix A.19.3 (p.59) as a part of Meta-Parameter where you can get the pointer to function parameters.
We'll gradually add all/most of the stuff pruned between N4451 and P0194R0 back to the rationale paper so that it's there to be discussed.

 

By the way, my understanding is that there will be a TS for reflection? Is it already ongoing?

Not yet AFAIK (Oulu will be the first time that I'll get (hopefully) to attend the meeting in person), but I hope that we will make some progress in Oulu in this regard.

 

 

3. Another possible extension which I think might be difficult but very very useful too would be a way to be able to

inject a call after or before another any call of a function.

This would enable, for example, the "all member functions will log" case by doing this:

Here we go into AOP territory, which is a whole different ball game, but I agree that it's related to reflection and would be nice to have.

 

Just in case this pov helps: I tend to see this idea like custom special functions, like destructors, because they are also

injected automatically in user's code. Maybe arguing that destructors is more or less a rigid precedent of this potential might help make a point.

Yes this is a interesting POV, since RAII usually already is the mechanism used to implement the function body entry and exit hooks in various AOP replacements / alternatives.

 
[8<]

I now remember reading about the identifier operator.

About 4.9 and 5.4.1, I'm not totally sure if there is a need for always generating identifier by passing through a string, even if the feature would help in other cases.

Maybe there could be a way to manipulate identifiers not a string but as something that compile-time-only that can take strings as concatenation and generate

an identifier? That way you would not need to have compile-time string facilities, only compiler "magic" (but that might be too much magic).

I'm not sure how to make an example of this for now, I'll think about it.

I'm starting to be a fan of the format-operator approach:

identifier("format_%1_%2%3", MetaNamed1, MetaNamed2, MetaNamed3)

described in 5.4.2. It is not so difficult to implement as the first one (from 5.4.1), yet it's still very powerful.

[8<]

[Klaim - Joël Lamotte]

On 23 April 2016 at 08:44, Matus Chochlik <choc...@gmail.com> wrote:

I now remember reading about the identifier operator.

About 4.9 and 5.4.1, I'm not totally sure if there is a need for always generating identifier by passing through a string, even if the feature would help in other cases.

Maybe there could be a way to manipulate identifiers not a string but as something that compile-time-only that can take strings as concatenation and generate

an identifier? That way you would not need to have compile-time string facilities, only compiler "magic" (but that might be too much magic).

I'm not sure how to make an example of this for now, I'll think about it.

I'm starting to be a fan of the format-operator approach:

identifier("format_%1_%2%3", MetaNamed1, MetaNamed2, MetaNamed3)

described in 5.4.2. It is not so difficult to implement as the first one (from 5.4.1), yet it's still very powerful.

[8<]

I agree that it would be very powerful.

Are there blocking points or critics from the last reviews of the proposal that will prevent or slow-down the adoption of this

or the other features? Why did this identifier operator was removed initially?

[Matus Chochlik]

Part of the feedback on N4451 was, that we should concentrate on showing that the basic principles (the reflection operator + the representation of metaobjects as minimalistic types) are solid and extensible, on the reflection of a limited subset of language features.
Some of the more advanced features from N4451 (and even several fairly trivial ones, like distinguishable typedefs or the reflection of private members) were meeting some opposition. So we are focusing on explaining our rationale and making the initial proposal acceptable for the majority of people, without having any really contentious issues and we can extend that in future proposals.

[Klaim - Joël Lamotte]

In my opinion, being able to generate new types at compile-time based on meta data is one of the basic features of such a reflection system.

Whithout this, the only actual thing we can do easily is serialization, which is already something, but clearly shows a massive gap of needed functionality.

So to me, generating identifiers should not be an potential extension but definitively required feature.

Maybe it will be more obvious to more people after playing with an implementation.

[Thiago Macieira]

On sábado, 23 de abril de 2016 21:50:51 PDT Klaim - Joël Lamotte wrote:
> > Part of the feedback on N4451 was, that we should concentrate on showing
> > that the basic principles (the reflection operator + the representation of
> > metaobjects as minimalistic types) are solid and extensible, on the
> > reflection of a limited subset of language features.
> > Some of the more advanced features from N4451 (and even several fairly
> > trivial ones, like distinguishable typedefs or the reflection of private
> > members) were meeting some opposition. So we are focusing on explaining
> > our
> > rationale and making the initial proposal acceptable for the majority of
> > people, without having any really contentious issues and we can extend
> > that
> > in future proposals.
>
> In my opinion, being able to generate new types at compile-time based on
> meta data is one of the basic features of such a reflection system.

In mine and in many others', it isn't. Hence, it's a point of contention. Why
not focus on what isn't contentious?

> Whithout this, the only actual thing we can do easily is serialization,
> which is already something, but clearly shows a massive gap of needed
> functionality.

My objective is to implement moc's functionality without an extra
preprocessor. That's actually a lot of ground covered there.

> So to me, generating identifiers should not be an potential extension but
> definitively required feature.

--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center

[Matus Chochlik]

As the author of the proposal I don't need any convincing that all these wonderful* ideas should be implemented (*but then, I might be biased ;)). The fact that this feature is needed for the implementation of one of the targeted use cases from the committee CFP is IMO reason enough that it /will/ be part of the reflection facility eventually (and the sooner, the better), even if not immediately.

I'm not a member of the committee, but by some people who *are* I've been told that it seems that a big part of the committee likes this proposal and is close to being convinced that "[...this] proposal can accommodate reflection of all of C++." On the other hand there are some disputed issues remaining and the committee does not like to stomp on minority opinions and tries to achieve consensus if possible (this has also been discussed here https://www.youtube.com/watch?v=PqU_ot4BlNQ).

So while being personally impatient too, I do think that showing and convincing as many people as possible, that we have a solid foundation (covering many use cases, not just single ones here and there), without messing up the rest of the core language, a solid plan for the future evolution, and a solid rationale for our design is a good way forward.

 

[Matus Chochlik]

On Sun, Apr 24, 2016 at 12:26 AM, Thiago Macieira <thi...@macieira.org> wrote:

On sábado, 23 de abril de 2016 21:50:51 PDT Klaim - Joël Lamotte wrote:
> > Part of the feedback on N4451 was, that we should concentrate on showing
> > that the basic principles (the reflection operator + the representation of
> > metaobjects as minimalistic types) are solid and extensible, on the
> > reflection of a limited subset of language features.
> > Some of the more advanced features from N4451 (and even several fairly
> > trivial ones, like distinguishable typedefs or the reflection of private
> > members) were meeting some opposition. So we are focusing on explaining
> > our
> > rationale and making the initial proposal acceptable for the majority of
> > people, without having any really contentious issues and we can extend
> > that
> > in future proposals.
>
> In my opinion, being able to generate new types at compile-time based on
> meta data is one of the basic features of such a reflection system.

In mine and in many others', it isn't. Hence, it's a point of contention. Why
not focus on what isn't contentious?

This feature is the requirement for the #2 use case on the list of targeted use cases from the original committee's Call for reflection proposals, so it IMO does warrant focus.

 

> Whithout this, the only actual thing we can do easily is serialization,
> which is already something, but clearly shows a massive gap of needed
> functionality.

My objective is to implement moc's functionality without an extra
preprocessor. That's actually a lot of ground covered there.

This is just a single, relatively niche use case of many, what about all the others. Do you have at least some plan how to cover them?

[Ricardo Andrade]

Hi Matus,

One additional question for your FAQ:

Why the reflexpr() operator needs to return different types instead of different constexpr *instances* of the same type?

My comments:

What goes inside of one or the other is the "compiler's black magic" to refer back to the original declaration anyway.

Instances of the same time would allow:

- Returning arrays instead of MetaSequence and constexpr functions could easily manipulate those.

- Additional constexpr data members, member functions and operators could be added for extra usability (std::to_string come to mind).

Ruminations:

It sounds like that would require less resources to compile, but I don't have a proof of it at this time.

Maybe the compiler magic needs to be "stronger" to inspect values instead of types?

Well, compiler know what is inside of each reference... it would be just a super-reference (to anything).

Regards,

Ricardo

[Matus Chochlik]

Hi Ricardo,

thanks for your feedback. I will incorporate it and the answers to your questions into the paper in the following days.

[Matus Chochlik]

Hi Ricardo,

On Sun, Apr 24, 2016 at 5:32 AM, Ricardo Andrade <ricardofabi...@gmail.com> wrote:

Hi Matus,

One additional question for your FAQ:

Why the reflexpr() operator needs to return different types instead of different constexpr *instances* of the same type?

My comments:

What goes inside of one or the other is the "compiler's black magic" to refer back to the original declaration anyway.

Instances of the same time would allow:

- Returning arrays instead of MetaSequence and constexpr functions could easily manipulate those.

- Additional constexpr data members, member functions and operators could be added for extra usability (std::to_string come to mind).

What about if every metaobject had its own compile-time constant identifier? For example a `std::uintptr_t` wrapped in a struct to make it distinguishable. This uid would be a homogeneous compile-time constant which could be stored in arrays, compared with `operator ==`, etc. and there would be two operations for the conversion between the type representation and the constexpr instances of uid.

Then the metaobjects could (but wouldn't have to) be internally implemented as:

``

namespace meta {

using uid_t = std::uintptr_t;

class identifier
{

private:

    uid_t _id;

public:

};

template <uint_t Id>

struct object { };

} //

[Ricardo Fabiano de Andrade]

On Mon, Apr 25, 2016 at 2:39 AM, Matus Chochlik <choc...@gmail.com> wrote:

Hi Ricardo,

On Sun, Apr 24, 2016 at 5:32 AM, Ricardo Andrade <ricardofabi...@gmail.com> wrote:

Hi Matus,

One additional question for your FAQ:

Why the reflexpr() operator needs to return different types instead of different constexpr *instances* of the same type?

My comments:

What goes inside of one or the other is the "compiler's black magic" to refer back to the original declaration anyway.

Instances of the same time would allow:

- Returning arrays instead of MetaSequence and constexpr functions could easily manipulate those.

- Additional constexpr data members, member functions and operators could be added for extra usability (std::to_string come to mind).

What about if every metaobject had its own compile-time constant identifier? For example a `std::uintptr_t` wrapped in a struct to make it distinguishable. This uid would be a homogeneous compile-time constant which could be stored in arrays, compared with `operator ==`, etc. and there would be two operations for the conversion between the type representation and the constexpr instances of uid.

Then the metaobjects could (but wouldn't have to) be internally implemented as:

``

namespace meta {

using uid_t = std::uintptr_t;

class identifier
{

private:

    uid_t _id;

public:

};

template <uint_t Id>

struct object { };

} //

 

Initially, it sounds like a great idea but it would be even better if there was no need for a type representation.

If the constant was sufficient to identify the declaration and to be passed in get_name<constant> for example.

Then, by using arrays, instead of for_each a plain range-for loop in a constexpr function could be used.

Are there any situations where a type would be better than a just constant?

 

Ruminations:

It sounds like that would require less resources to compile, but I don't have a proof of it at this time.

Maybe the compiler magic needs to be "stronger" to inspect values instead of types?

Well, compiler know what is inside of each reference... it would be just a super-reference (to anything).

--
You received this message because you are subscribed to a topic in the Google Groups "SG 7 - Reflection" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/reflection/iOalIdr6rxw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to reflection+...@isocpp.org.

[Matus Chochlik]

On Tue, Apr 26, 2016 at 8:30 AM, Ricardo Fabiano de Andrade <ricardofabi...@gmail.com> wrote:

One additional question for your FAQ:

Why the reflexpr() operator needs to return different types instead of different constexpr *instances* of the same type?

My comments:

What goes inside of one or the other is the "compiler's black magic" to refer back to the original declaration anyway.

Instances of the same time would allow:

- Returning arrays instead of MetaSequence and constexpr functions could easily manipulate those.

- Additional constexpr data members, member functions and operators could be added for extra usability (std::to_string come to mind).

What about if every metaobject had its own compile-time constant identifier? For example a `std::uintptr_t` wrapped in a struct to make it distinguishable. This uid would be a homogeneous compile-time constant which could be stored in arrays, compared with `operator ==`, etc. and there would be two operations for the conversion between the type representation and the constexpr instances of uid.

Then the metaobjects could (but wouldn't have to) be internally implemented as:

``

namespace meta {

using uid_t = std::uintptr_t;

class identifier
{

private:

    uid_t _id;

public:

};

template <uint_t Id>

struct object { };

} //

 

Initially, it sounds like a great idea but it would be even better if there was no need for a type representation.

If the constant was sufficient to identify the declaration and to be passed in get_name<constant> for example.

Then, by using arrays, instead of for_each a plain range-for loop in a constexpr function could be used.

Replacing metaobject sequences with arrays of (eagerly generated) metaobjects would actually be *much* less efficient in some cases. Also there are limits to what you can do in constexpr functions.

 

Are there any situations where a type would be better than a just constant?

Interaction with type traits and the heaps of legacy metaprogramming code. OK, we now have Boost.Hana & constexpr metaprogramming. which are wonderful, but that doesn't mean that template metaprogramming gets thrown out of the window.

This is again the thing about having more options.

Furthermore, nothing in P0194 prevents the metaobjects to be internally represented as constants (if the compiler vendors come to the conclusion that this is the best option) and the whole type representation would consist of this *single* line of code:

``template <uint_t Id>struct __metaobject;``

Note that the type only needs to be forward declared but not defined. But this may not always be the best way to represent metaobjects so I don't want to make such implementation mandatory.

[Ricardo Fabiano de Andrade]

On Tue, Apr 26, 2016 at 2:01 AM, Matus Chochlik <choc...@gmail.com> wrote:

On Tue, Apr 26, 2016 at 8:30 AM, Ricardo Fabiano de Andrade <ricardofabi...@gmail.com> wrote:

One additional question for your FAQ:

Why the reflexpr() operator needs to return different types instead of different constexpr *instances* of the same type?

My comments:

What goes inside of one or the other is the "compiler's black magic" to refer back to the original declaration anyway.

Instances of the same time would allow:

- Returning arrays instead of MetaSequence and constexpr functions could easily manipulate those.

- Additional constexpr data members, member functions and operators could be added for extra usability (std::to_string come to mind).

What about if every metaobject had its own compile-time constant identifier? For example a `std::uintptr_t` wrapped in a struct to make it distinguishable. This uid would be a homogeneous compile-time constant which could be stored in arrays, compared with `operator ==`, etc. and there would be two operations for the conversion between the type representation and the constexpr instances of uid.

Then the metaobjects could (but wouldn't have to) be internally implemented as:

``

namespace meta {

using uid_t = std::uintptr_t;

class identifier
{

private:

    uid_t _id;

public:

};

template <uint_t Id>

struct object { };

} //

 

Initially, it sounds like a great idea but it would be even better if there was no need for a type representation.

If the constant was sufficient to identify the declaration and to be passed in get_name<constant> for example.

Then, by using arrays, instead of for_each a plain range-for loop in a constexpr function could be used.

Replacing metaobject sequences with arrays of (eagerly generated) metaobjects would actually be *much* less efficient in some cases. Also there are limits to what you can do in constexpr functions.

Not metaobjects, I'm suggesting an array of uintptr_t constants.

More specifically:

``

template <size_t N>

std::array<uid_t, N> get_all_data_members_v;

``

Is this less efficient than returning a type list (meta sequence)?

 

 

Are there any situations where a type would be better than a just constant?

Interaction with type traits and the heaps of legacy metaprogramming code. OK, we now have Boost.Hana & constexpr metaprogramming. which are wonderful, but that doesn't mean that template metaprogramming gets thrown out of the window.

Agreed. This is a compile-time solution, metaprogramming is a must.

 

This is again the thing about having more options.

I don't see how that eliminates the option of metaprogramming.

Based on the example above, this would a possible situation:

``

#include <reflexpr>

#include <utility>

template <int ...values>

struct dummy{

    int a[sizeof...(values)] = { values... };

};

template<typename T, std::size_t N>

constexpr auto array_size(const T (&t)[N]) {

    return N;

}

template <typename T, const T& t, std::size_t ...I>

constexpr auto to_dummy(std::index_sequence<I...>) {

    return dummy<std::get<I>(t)...>{};

}

template <typename T, const T& t>

constexpr auto to_dummy() {

    return to_dummy<T, t>(std::make_index_sequence<array_size(t)>{});

}

extern constexpr auto uids = std::meta::get_all_data_members_v<reflexpr(T)>;

constexpr auto ddd = to_dummy<decltype(uids), uids>();

``

A little bit ugly but have in mind that ``<decltype(uids), uids>`` will be deduced by ``template <auto>`` once P0127R0 gets into the standard (seems likely).

I'm not sure if there's anything that can be done about the need of ``extern``, unless ``get_all_data_members_v``` by itself has such linkage.

In this case, it could work like:

``

to_dummy<std::meta::get_all_data_members_v<reflexpr(T)>>();

``

The point is... arrays make the life easier in constexpr functions without sacrificing (too much) metaprogramming.

I'll only pay the price of metaprogramming if I use it.

 

Furthermore, nothing in P0194 prevents the metaobjects to be internally represented as constants (if the compiler vendors come to the conclusion that this is the best option) and the whole type representation would consist of this *single* line of code:

``template <uint_t Id>struct __metaobject;``

This is great but it could be used just when/if needed, instead of being the result of reflexpr().

 

Note that the type only needs to be forward declared but not defined. But this may not always be the best way to represent metaobjects so I don't want to make such implementation mandatory.

I may be missing something but it still fells like as constants could be used directly.

If I'm really missing something, what's your take on using value lists?

``

template <int ...values>

struct get_all_data_members_t;

``

Cheers,

Ricardo

[Matus Chochlik]

On Thu, Apr 28, 2016 at 6:53 AM, Ricardo Fabiano de Andrade <ricardofabi...@gmail.com> wrote:

Initially, it sounds like a great idea but it would be even better if there was no need for a type representation.

If the constant was sufficient to identify the declaration and to be passed in get_name<constant> for example.

Then, by using arrays, instead of for_each a plain range-for loop in a constexpr function could be used.

Replacing metaobject sequences with arrays of (eagerly generated) metaobjects would actually be *much* less efficient in some cases. Also there are limits to what you can do in constexpr functions.

Not metaobjects, I'm suggesting an array of uintptr_t constants.

More specifically:

``

template <size_t N>

std::array<uid_t, N> get_all_data_members_v;

``

And how would you then distinguish between a plain old uintptr_t constant and a metaobject?

 

Is this less efficient than returning a type list (meta sequence)?

The array requires the compiler to scan the whole list of member declarations and generate (and store internally some representation of) a constant for each of these declarations *eagerly*. When using a metaobject sequence the sequence is a *single* object regardless of the number of elements. The elements are instantiated only when explicitly queried.

 

 

 

Are there any situations where a type would be better than a just constant?

Interaction with type traits and the heaps of legacy metaprogramming code. OK, we now have Boost.Hana & constexpr metaprogramming. which are wonderful, but that doesn't mean that template metaprogramming gets thrown out of the window.

Agreed. This is a compile-time solution, metaprogramming is a must.

Also we have ~15 years of experience with template metaprogramming, and much more legacy and even new MP code still uses this paradigm. Constexpr metaprogramming is just taking off.

Also, constexpr values can  'decay' to runtime values, so you may have interactions with the linker.

What happens if I invent my own constexpr value like 42 and feed that into one of the reflection functions? Since you don't distinguish the metaobject on type level the compiler will need to remember all metaobjects which it has generated. With types you could in some expressions generate the metaobject use it to get metadata and then throw it away.

 

 

This is again the thing about having more options.

I don't see how that eliminates the option of metaprogramming.

Based on the example above, this would a possible situation:

``

#include <reflexpr>

#include <utility>

template <int ...values>

struct dummy{

    int a[sizeof...(values)] = { values... };

};

again, this must be eagerly initialized.

 

template<typename T, std::size_t N>

constexpr auto array_size(const T (&t)[N]) {

    return N;

}

template <typename T, const T& t, std::size_t ...I>

constexpr auto to_dummy(std::index_sequence<I...>) {

    return dummy<std::get<I>(t)...>{};

}

template <typename T, const T& t>

constexpr auto to_dummy() {

    return to_dummy<T, t>(std::make_index_sequence<array_size(t)>{});

}

extern constexpr auto uids = std::meta::get_all_data_members_v<reflexpr(T)>;

constexpr auto ddd = to_dummy<decltype(uids), uids>();

``

A little bit ugly but have in mind that ``<decltype(uids), uids>`` will be deduced by ``template <auto>`` once P0127R0 gets into the standard (seems likely).

I'm not sure if there's anything that can be done about the need of ``extern``, unless ``get_all_data_members_v``` by itself has such linkage.

Meanwhile P0194 has no interactions with the linker at the moment.

 

In this case, it could work like:

``

to_dummy<std::meta::get_all_data_members_v<reflexpr(T)>>();

``

The point is... arrays make the life easier in constexpr functions without sacrificing (too much) metaprogramming.

And how do you know that the not too many use cases you are willing to sacrifice are not important for someone else?
 

I'll only pay the price of metaprogramming if I use it.

Using thinks like arrays and values actually, you can actually pay more that with types.

 

 

 

Furthermore, nothing in P0194 prevents the metaobjects to be internally represented as constants (if the compiler vendors come to the conclusion that this is the best option) and the whole type representation would consist of this *single* line of code:

``template <uint_t Id>struct __metaobject;``

This is great but it could be used just when/if needed, instead of being the result of reflexpr().

Again ~fifteen years of TMP vs. a couple of years (at most) with constexpr MP. Why not start with something we are much more familiar with? We can move on with other approaches one the dust and the initial euphoria about shiny new paradigms settles and its side effects a quirks are better understood?

 

 

Note that the type only needs to be forward declared but not defined. But this may not always be the best way to represent metaobjects so I don't want to make such implementation mandatory.

I may be missing something but it still fells like as constants could be used directly.

If I'm really missing something, what's your take on using value lists?

``

template <int ...values>

struct get_all_data_members_t;

``

 

in addition to what I've said above, we do need compile-time metaprogramming. And yes TMP can look weird and complicated, on the other hand the additional rules which must be obeyed so that CT values stay constexpr make this form of MP equally complex.

Also saying that metaobjects must be constants you are IMHO limiting the options what the compiler implementers can do with the implementation.

 

Matus

[Ricardo Fabiano de Andrade]

Well, those were convincing arguments :)

The additional explanation is really appreciated.

I guess it's more a question of seeing it in action now.

It may have comments on some interfaces which will address in a separate post.

--

[Roland Bock]

Dear Matus,

Thanks for providing the document! This is definitely going in a
direction I like a lot :-)

The one thing that I am missing so far is variadic composition, which I
describe here: https://github.com/rbock/reasoning-about-names

With this you could get around the variadic inheritance you are
currently using to add members to the soa_struct:

```C++
// ---------------------------------------

template <typename ... MetaDataMembers>
struct soa_compose_all
{
vector<
meta::get_original_type_t<meta::get_type_t<MetaDataMembers>>
>... identifier(meta::get_name_v<MetaDataMembers>);
};


template <typename T>
using soa = meta::unpack_sequence_t<
meta::get_data_members_t<reflexpr(T)>,
soa_compose_all>;

// ---------------------------------------
```

Shorter, easier to grok (IMHO), no inheritance, no base "member" type.

For libraries like sqlpp11 that would be a game changer. It would also
allow for using SQL on C++ containers elegantly.

What do you think?

Cheers,

Roland

> --
> You received this message because you are subscribed to the Google

> Groups "SG 7 - Reflection" group.

> To unsubscribe from this group and stop receiving emails from it, send
> an email to reflection+...@isocpp.org

> <mailto:reflection+...@isocpp.org>.

> To post to this group, send email to refle...@isocpp.org

> <mailto:refle...@isocpp.org>.

[Matus Chochlik]

Hi Roland,

On Tue, May 3, 2016 at 12:22 PM, Roland Bock <rb...@eudoxos.de> wrote:

Dear Matus,

Thanks for providing the document! This is definitely going in a
direction I like a lot :-)

Thanks for your kind words :)

 

The one thing that I am missing so far is variadic composition, which I
describe here: https://github.com/rbock/reasoning-about-names

Agreed, variadic composition would be a very useful feature together with the ability to generate identifiers. I will add a couple of sentences discussing this in the "Future" section.

 

With this you could get around the variadic inheritance you are
currently using to add members to the soa_struct:

```C++
// ---------------------------------------

template <typename ... MetaDataMembers>
struct soa_compose_all
{
  vector<
    meta::get_original_type_t<meta::get_type_t<MetaDataMembers>>
  >... identifier(meta::get_name_v<MetaDataMembers>);
};


template <typename T>
using soa = meta::unpack_sequence_t<
               meta::get_data_members_t<reflexpr(T)>,
               soa_compose_all>;

// ---------------------------------------
```

Shorter, easier to grok (IMHO), no inheritance, no base "member" type.

For libraries like sqlpp11 that would be a game changer. It would also
allow for using SQL on C++ containers elegantly.

Matus

[Roland Bock]

On 2016-05-03 13:29, Matus Chochlik wrote:
> Hi Roland,
>
> On Tue, May 3, 2016 at 12:22 PM, Roland Bock <rb...@eudoxos.de
> <mailto:rb...@eudoxos.de>> wrote:
>
> Dear Matus,
>
> Thanks for providing the document! This is definitely going in a
> direction I like a lot :-)
>
>
> Thanks for your kind words :)
>
>
>
> The one thing that I am missing so far is variadic composition, which I
> describe here: https://github.com/rbock/reasoning-about-names
>
>
> Agreed, variadic composition would be a very useful feature together
> with the ability to generate identifiers. I will add a couple of
> sentences discussing this in the "Future" section.

Thanks :-)

Speaking of which: Does your experimental version of clang support
generating identifiers already? I'd really like to experiment with that.

Best,

Roland

[Matus Chochlik]

>

> Agreed, variadic composition would be a very useful feature together
> with the ability to generate identifiers. I will add a couple of
> sentences discussing this in the "Future" section.

Thanks :-)

Speaking of which: Does your experimental version of clang support
generating identifiers already? I'd really like to experiment with that.

Unfortunately no. At the moment I'm busy with other things. I have a plan that I'll start a new implementation, including some of the new features, from scratch in the summer. But I'm also seriously considering a job change, so I don't want to make any big promises yet.

BR,

Matus