Augment the Preprocessor to Get Rid of Mores Uses of the Preprocessor

[Ricardo Fabiano de Andrade]

Motivation

Over the years, there has been a lot of arguments against the Preprocessor and supporting decades of preprocessor-based code is sometimes a daunt task but in the current state of the standard one can hardly live without it.

In the other hand, it's hard to deny the usefulness of the Preprocessor when it comes to purely mechanical tasks involving code generation, being the extreme example of this the Boost Preprocessor library.

Interoperability with C is also a point in favor of keeping the Preprocessor around.

C++ has got many tools to avoid using the Preprocessor though, namely: constants, templates, inline, constexpr, user-defined literals, at some capacity modules and soon, reflection.

This last one has the potential to, in the future, eliminate almost all uses of the Preprocessor, except by fact that any reflection facility added to the language would operate after the work of the Preprocessor is done.

Having this in mind and knowing that almost any piece of legacy code will contain an abundant number of macros, it's concerning the fact that the language support for reflection will not be able to reason about the Preprocessor tokens which may be declaring the objects and functions that are actually making up all the APIs of such piece of code.

Conversely, it's hard to imagine a resulting C++ language feature, reflection or otherwise, which would allow crossing the Preprocessor boundary to obtain information about its macros declarations in a clean way.

Instead, the standard should be changed in the direction to augment the Preprocessor capabilities in a way to provide some support for introspection and that is what this proposal is about.

Preprocessor Minimal Introspection

The Preprocessor already has some pseudo-reflection facilities in the form of the # (strigizing) and ## (token-pasting) operators.

However, it does not provide any introspection support. For example, operations to list the macros defined or to tell if a given macro takes arguments are not present.

The Macro Listing Operation

This is an operation which lists the declared macros. One may think that such operation should be contained in an operator such as the ones mentioned above but the problem is that there are no reserved characters which could be used by a new operator in the Preprocessor and using an operator would prevent this operation from taking arguments.

Therefore it is proposed to have a function-like internal macro tentatively named: __DEFINED_MACROS__(filter)

Being "filter" a string literal (more below).

The returned value is the following:

- If the filter matches anything, the result is the comma-separated macro names.

- If no matches are found, the result is nothing.

Macro List Filtering

The number of macros even in a small program can be overwhelming, so to make the resulting of this operation more digestible it is proposed that it supports filtering.

The author of this proposal would like to have regular expressions as the filter mechanism but realistically speaking it would be very difficult to reach an agreement on which variant (posix, ECMA, grep, etc.) should be supported or how to select one or define options.

For now, this proposal will try to stick with the widely accepted wildcard character '*' (any multiple characters) and '?' (any single character) and the known mechanisms associated with those in the bash "globbing". That should be sufficient for the majority of the use cases.

For example, using the following snippet (extracted from OpenLDAP ldap.h):

#define LDAP_DEBUG_TRACE	0x001
#define LDAP_DEBUG_PACKETS	0x002
#define LDAP_DEBUG_ARGS		0x004

The call below:

__DEFINED_MACROS__("LDAP_DEBUG_*")

Would have the result of (incomplete for brevity):

LDAP_DEBUG_TRACE, LDAP_DEBUG_PACKETS, LDAP_DEBUG_ARGS ...

Which in turn could be input for (non working, just illustrative):

#define FOR_EACH(function, ...) // out of the scope of this proposal

#define DECLARE_ENUM(macro) e#macro = macro,

// Redeclare LDAP debug constants as a strong-typed enum.

enum class LdapDebug {

FOR_EACH(DECLARE_ENUM, __DEFINED_MACROS__("LDAP_DEBUG_*"));

}; // aware of the trailing comma, please ignore it

Now, LdapDebug names and values could be used in C++ and also available to reflection.

This might work great for macros which declare objects, what about functions?

The Macro Arguments Operations

The first operation returns the number of arguments, the second their names. Again, internal function-like pre-defined macros will be proposed instead of new operators.

The number of arguments of a macro would be obtained with (tentative name):

__DEFINED_ARGS_COUNT__(macro)

The name of the arguments could be obtained with (tentative name):

__DEFINED_ARGS_NAMES__(macro)

Being "macro" the string literal representing the macro name (# operator can be used).

The returned value is the following:

- If a function macro name is passed, the result is the number of arguments (for ...COUNT) or their comma-separated names (for ...NAMES).

- If a non-function macro name is passed, the result is nothing.

Please note that __VA_ARGS__ is counted as a regular argument, and its name will be present among other argument names in the result.

For example (again, from ldap.h again):

#define Debug( level, fmt, arg1, arg2, arg3 ) ...

Could be transformed into a lambda (non working, just illustrative):

#define TEMPLATE_ARG(name) auto #name,
#define LDAP_FUNC_DECLARE(macro) auto = Ldap#macro[]( \
FOR_EACH(TEMPLATE_ARG, __DEFINED_ARGS_NAMES__(#macro))) \  
{ return #macro(__DEFINED_ARGS_NAMES__(#macro)); }
// aware of the trailing comma, please ignore it

// Redeclare LDAP functions as generic lambdas.
#if defined(__DEFINED_ARGS_COUNT__(#Debug))
LDAP_FUNC_DECLARE(Debug);
#endif

The lambda could be then be made available for C++ and even though the current reflection proposal (P0194R2) would not be able to handle it, the authors of that proposal made clear their intentions to support functions at some point in the future.

Conclusion

Stretch this ideas of this proposal a little bit and you can vision definitions being redeclared this way and used in C++ instead of the legacy macros which in turn could get deprecated over time (of course, if C support is to be dropped). The author of this proposal can see derivations of such work aiding with a better support modules in legacy environments too.

Please note that C++ is the language the benefit the most from the changes proposed here but nothing prevents such ideas to be made also available for C, if the concern is having the same Preprocessors for C and C++.

I'm looking forward to hearing comments and suggestions.

If such functionality can already be achieved using already available techniques, I'd love to hear more.

Thank you,

Ricardo Andrade