Musings: generalized metaclasses/multi-stage compile-time programming.

[Anthony Hall]

(With apologies to those who will have already seen these musings in the #future_standard channel of cpplang.slack)

There’s an interesting metaprogramming language/project called Terra that I think represents the fully general case of what metaclasses and the current C++ compile-time programming proposals are working towards, but with Lua as the meta-language and a new low-level, JIT-compiled language (called Terra) as the “concrete” language.  It builds on the notion of multi-stage programming (which I understand to mean that one compilation step can be programmed in a way that controls a deeper compilation step, which can in turn return first-class compilation metadata to the outer step).  http://terralang.org/ (See particularly the Generative Programming section)

I’ve spent some time wondering if C++ compile-time programming could be done in a similar way — with `constexpr` being generalized more and more, we almost have what we would need to use C++ as its own metalanguage.

I think it would be a long, long shot to hope for the standard to mandate the ability to JIT C++ as the “concrete” layer of execution, so this direction in C++ still wouldn’t have the full run-time generative power of Terra — All Terra programs are actually Lua programs that are run like normal Lua programs, but it JITs the Terra portions then executes them as it runs.  In C++ we have to have a compiler itself run (interpret) the meta/constexpr portions.

But one thing I like about Terra is the very terse syntax for quoting Lua in Terra code, and vice-versa.  Seeing `constexpr {...}` blocks in the metaclasses paper reminded me of backtick-quoting in the Lua code of a Terra program.  Terra code itself can also invoke meta-level Lua code by quoting with `[…]`, which reminds me of metaclasses’ `->`-injections

Anyway, what’s being done with metaclasses reminded me of what I’d learned previously about multi-stage compilation and Terra in particular; I thought a pointer to another project in a similar vein might be interesting to others as another reference point.

- - -

I do like how terse `-> {...}` code injections are in the metaclasses proposal.  I wonder how strong interest would be in making `constexpr {...}` terser.  Maybe something like `<- {...}`?  The symmetry of that notation with code injections might be nice as a visual mnemonic, if notions such as meta-metaclasses are supported…

 

I wonder if the generality of having a notion of a full stack of meta-levels would be useful, or if there’s enough power in only one meta-level.

I’m picturing that with a full stack, it could be handy to be able to escape to a meta-meta level which can use code insertions to operate upon the single-meta level.  An insertion inside that insertion would then take it back down to the concrete level.  Generally, the default level is the concrete level, and we use `<-{…}` to go up one meta-level.

-Andy

[Bengt Gustafsson]

Could you please provide a link to the metaclasses paper you are referring to?

[Jackie Kay]

P0707 by Herb Sutter: https://www.google.co.uk/url?sa=t&source=web&rct=j&url=http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0707r0.pdf&ved=0ahUKEwjOmvOf0ezUAhVfFMAKHfWWD6YQFghdMAU&usg=AFQjCNGBH-aFb0DGA7dVN8APrHqfbuZVKQ

--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/c0f7f7fc-545b-458f-95a1-0e36ee1a82e7%40isocpp.org.

[Bengt Gustafsson]

I find this proposal very interesting, as it offers a lot towards getting rid of the preprocessor, and increases expressiveness of C++ significantly, especially when using it to implement DSLs for different purposes.

In the Qt properties example each property which just needs the default getter/setter functions still has a empty class head {} attached. I was thinking that the forward declaration of a property could somehow be enough to trigger some version of the constexpr code that generates the getter/setter methods, so that the {} can be omitted.

I wonder whether the $class introducer could be replaced by inheriting from class (or some other metaclass that in itself inherits from class):

class safe_union : class {

};

I would also like to see this extended to other parts of the C++ language, at least statement, so that a safe switch, and other fancy control structures can be defined. This would however require some kind of pattern handling to describe what is supposed to follow the keyword itself. Not to disrupt the parser too much this must of course be limited to a few select constructs, basically the ones we already have, including: class-head, statement-body, for-init-statement. This line of thinking leads to the (probably quite old) idea of viewing this type of construct as constexpr functions returning code. Assuming this stuff is in  a meta namespace we would get something like:

constexpr meta::code safe_union(meta::class_head head) { ... }

constexpr meta::code safe_switch(meta::paren_expression, meta::compound_statement) { ... }

Of course this would be much more comprehensive than the proposal and more work to get done, but if this is an interesting direction it would be a bad idea to choose a syntax now that basically prevents extension (in a consistent way) in this direction. By this I mean to say that the meta namespace does not have to have much contents in the first iteration, only enough to implement the original proposal, but the constexpr function way of description offers an easy extension path for future versions. Also it does away with the need for a specai $ marker (for better or worse).

I would love my proposed for-expression (P0565) to be expressible using this system. This could be used as a litmus test of something we would like to be able to define.

[Matthew Woehlke]

On 2017-07-03 03:55, Jackie Kay wrote:
> P0707 by Herb Sutter:
> https://www.google.co.uk/url?sa=t&source=web&rct=j&url=http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0707r0.pdf&ved=0ahUKEwjOmvOf0ezUAhVfFMAKHfWWD6YQFghdMAU&usg=AFQjCNGBH-aFb0DGA7dVN8APrHqfbuZVKQ

Somewhat off topic (more general discussion of that paper, not really
related to anything in the OP), but I'm concerned by how little
knowledge of Qt's MOC is displayed in P0707. In particular, it gives
these examples of properties:

Q_PROPERTY(int value READ get_value WRITE set_value)
int get_value() const { return value; }
void set_value(int v) { value = v; }

...but this is NOT AT ALL what real properties in Qt look like.

Here is a real example:

Q_PROPERTY(bool readOnly READ isReadOnly WRITE setReadOnly)

Note that NEITHER the getter nor setter have names that are trivially
derived (e.g. string concatenation). This information needs to be
communicated somehow. It's also not obvious how the `property` results
in injection of methods into the parent class.

The *idea* of P0707 is interesting, but until someone can *demonstrate*
— meaning, an actual functioning demo with real code, not some
ignorance-revealing hand-waving — that it can replace MOC, I'm not
convinced that it is a viable, complete proposal. At the very least, I
feel like the MOC example should be removed, since it is really
something of a lie.

Another problem is locality of code. Due to compatibility constraints,
it is *critical* that the implementation of many functions is
non-inline. Maybe modules will alleviate or solve this, but until that
happens, the only way P0707 stands a chance of replacing MOC is to know,
somehow, if it should generate code because it is in the class's
implementation TU, or if it should generate only prototypes. Possibly I
missed it(? ...it's a long paper, after all), but I don't recall seeing
anything to this effect in the proposal.

I'd like to see, for example, how the proposed tools would implement this:

qobject QLineEdit : public QWidget
{
qproperty<bool> modified{
.read = isModified,
.write = setModified
};
qproperty<QString> text{
.read = text,
.write = setText,
.notify = textChanged
};
// ...

public:
bool isModified() const; // note: NOT inline!
void setModified(bool); // note: NOT inline!
// ...
void setText(QString);
void setText(/*... some overload ...*/);

public: // signals
qsignal textModified();
};

Besides the issue of where certain special function definitions reside,
as discussed above, some issues that this would need to address that
P0707 does not obviously cover include:

- Implicit inheritance when the class is already a descendant of
QObject. (Practically speaking, the above example is inaccurate; there
would almost certainly be a $qwidget meta-class in addition to $qobject,
but I've written it that way to demonstrate this issue.) Presumably
reflection will be able to sort this out, but it seems important enough
to not just gloss over.

- Terse syntax for linking a meta-object (note: the $qproperty must
result in code generation, but otherwise must eventually disappear from
the class without creating members) to other members of the enclosing
class. I've used named initializers as the most terse but still
syntactically valid construct that came to mind. Moving the definitions
of the get/set functions into the $qproperty is not appropriate; in most
cases, they cannot be inline due to PIMPL, and writing thunks is
pointless repetition. The approach shown is MUCH more palatable than
what is presented in P0707. (Ideally, this should also be able to accept
not only an identifier which is a member function of the parent class,
but one with a specific signature, without choking and dying in the face
of overloads. However, use of a string literal might be palatable;
presumably code generation can turn it back into an identifier.)

--
Matthew

[José Rodrigo]

I think, it's time to C++ have anotations.

[Nicol Bolas]

On Friday, July 14, 2017 at 3:36:08 PM UTC-4, Matthew Woehlke wrote:

On 2017-07-03 03:55, Jackie Kay wrote:
> P0707 by Herb Sutter:
> https://www.google.co.uk/url?sa=t&source=web&rct=j&url=http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0707r0.pdf&ved=0ahUKEwjOmvOf0ezUAhVfFMAKHfWWD6YQFghdMAU&usg=AFQjCNGBH-aFb0DGA7dVN8APrHqfbuZVKQ

Somewhat off topic (more general discussion of that paper, not really
related to anything in the OP), but I'm concerned by how little
knowledge of Qt's MOC is displayed in P0707. In particular, it gives
these examples of properties:

  Q_PROPERTY(int value READ get_value WRITE set_value)
  int  get_value() const { return value; }
  void set_value(int v)  { value = v; }

...but this is NOT AT ALL what real properties in Qt look like.

Here is a real example:

  Q_PROPERTY(bool readOnly READ isReadOnly WRITE setReadOnly)

Note that NEITHER the getter nor setter have names that are trivially
derived (e.g. string concatenation). This information needs to be
communicated somehow. It's also not obvious how the `property` results
in injection of methods into the parent class.

The *idea* of P0707 is interesting, but until someone can *demonstrate*
— meaning, an actual functioning demo with real code, not some
ignorance-revealing hand-waving — that it can replace MOC, I'm not
convinced that it is a viable, complete proposal. At the very least, I
feel like the MOC example should be removed, since it is really
something of a lie.

It's not a "viable, complete proposal", and I don't think the authors see it that way. Even the MOC section is specifically called out as being a "(sketch)". The goal of P0707 is to be moving in a direction where you can eventually replace MOC. But I don't think anyone involved with that paper believes that the proposed changes alone are sufficient to get the job done.

Another problem is locality of code. Due to compatibility constraints,
it is *critical* that the implementation of many functions is
non-inline. Maybe modules will alleviate or solve this,

Well, modules is gonna happen, and they're gonna happen a lot sooner than any form of reflection (standard committee craziness notwithstanding). Modules is almost a TS at this point, with two independent implementations. Static introspective reflection is still in the gestation stage.

So continuing to design features for a pre-modules world is not a good place to start.