optional va_args argument

[jacobnavia]

This code
void SetErrorN(ErrorCode ec, int mc,
const char* emf = NULL, va_list args = NULL);

was compiling OK until 2015 or so. It still compiles OK in my macintosh
using clang.

But in an arm 64 architecture it gives me

EventLog.h:16:61: error: could not convert ‘0l’ from ‘long int’ to
‘va_list {aka __va_list}’

How could I fix that?

There are MANY calls to those functions all over the place.

[Melzzzzz]

On 2017-07-19, jacobnavia <ja...@jacob.remcomp.fr> wrote:
> This code
> void SetErrorN(ErrorCode ec, int mc,
> const char* emf = NULL, va_list args = NULL);
>
> was compiling OK until 2015 or so. It still compiles OK in my macintosh
> using clang.
>
> But in an arm 64 architecture it gives me
>
> EventLog.h:16:61: error: could not convert ‘0l’ from ‘long int’ to
> ‘va_list {aka __va_list}’
>
> How could I fix that?

Try with: args = va_list()?

>
> There are MANY calls to those functions all over the place.

Only one definition I hope ;)


--
press any key to continue or any other to quit...

[jacobnavia]

Thanks, I did not know that the constructor would yield the equivalent
of an empty list.

[Scott Lurndal]

jacobnavia <ja...@jacob.remcomp.fr> writes:
>This code
>void SetErrorN(ErrorCode ec, int mc,
> const char* emf = NULL, va_list args = NULL);

It is not legal to set a default argument value for va_list,
which is an opaque implementation defined type.

[Öö Tiib]

Yes initializing with NULL is not quaranteed to work but not because
va_list is opaque. It is required to be complete object type that must
be default-constructible so va_list() must work as default argument.

[jacobnavia]

It may be not legal now, but it compiled without any problem until 2016
for at least a dozen of years.

Thanks to Mr Melzzzz I know now that the NULL argument must be replaced
by va-list(), i.e. a call to some constructor of a default value.

This is provoked by a change in the architecture: arm64 has a LOT of
registers and almost never the stack is used anymore for that purpose.

How it happens now when that default argument is passed anyway
explicitely is not clear. I suppose the default assignment is discarded
and the variable has the actually passed in value.

Luckily I haven't seen any comparisons with NULL in that code for that
"args" variable.

I suppose that now any comparison (test of) NULL will not work, so that
the comparisons should be done with the default value. But the
programmer has not any access to the default value when an actual value
is passed. NULL was a practical default value. Maybe there is an
overload of va_list with NULL comparisons?

Software is made of these ever deepening problems...

[Ian Collins]

On 07/20/17 08:24 AM, jacobnavia wrote:
> Le 19/07/2017 à 21:00, Scott Lurndal a écrit :
>> jacobnavia <ja...@jacob.remcomp.fr> writes:
>>> This code
>>> void SetErrorN(ErrorCode ec, int mc,
>>> const char* emf = NULL, va_list args = NULL);
>>
>> It is not legal to set a default argument value for va_list,
>> which is an opaque implementation defined type.
>>
>
> It may be not legal now, but it compiled without any problem until 2016
> for at least a dozen of years.

By luck!

> Thanks to Mr Melzzzz I know now that the NULL argument must be replaced
> by va-list(), i.e. a call to some constructor of a default value.

As Öö Tiib pointed out above, va_list is a default constructable
complete object type. "va-list()" creates a default constructed va_list.

> This is provoked by a change in the architecture: arm64 has a LOT of
> registers and almost never the stack is used anymore for that purpose.
>
> How it happens now when that default argument is passed anyway
> explicitely is not clear. I suppose the default assignment is discarded
> and the variable has the actually passed in value.

The args parameter *is* passed by value!

> Luckily I haven't seen any comparisons with NULL in that code for that
> "args" variable.

Probably because the "emf" parameter is a format string. No format
string, no args. I guess you could hit a snag if the there was a simple
string and no args such as "fred"...

> I suppose that now any comparison (test of) NULL will not work, so that
> the comparisons should be done with the default value. But the
> programmer has not any access to the default value when an actual value
> is passed. NULL was a practical default value. Maybe there is an
> overload of va_list with NULL comparisons?

Pass a va_list* instead? Or use a variadic template?

--
Ian

[Gareth Owen]

jacobnavia <ja...@jacob.remcomp.fr> writes:

> Le 19/07/2017 à 21:00, Scott Lurndal a écrit :
>> jacobnavia <ja...@jacob.remcomp.fr> writes:
>>> This code
>>> void SetErrorN(ErrorCode ec, int mc,
>>> const char* emf = NULL, va_list args = NULL);
>>
>> It is not legal to set a default argument value for va_list,
>> which is an opaque implementation defined type.
>>
>
> It may be not legal now, but it compiled without any problem until
> 2016 for at least a dozen of years.

That's how opaque types work. If you guess their type correctly for a
particular compiler, the compiler won't complain, but its not portable
(as you've discovered). As you well know, in C there's a big difference
between something that compiles & works in C or C++, and something that
is actually defined as legal.

[Bo Persson]

One option to default arguments is to add an overload with fewer
paramters, like:

void SetErrorN(ErrorCode ec, int mc);


which just sets the error code with no additional message.


Bo Persson

[Manfred]

On 7/19/2017 10:44 PM, Ian Collins wrote:
> On 07/20/17 08:24 AM, jacobnavia wrote:
>> This is provoked by a change in the architecture: arm64 has a LOT of
>> registers and almost never the stack is used anymore for that purpose.
>>
>> How it happens now when that default argument is passed anyway
>> explicitely is not clear. I suppose the default assignment is discarded
>> and the variable has the actually passed in value.
>
> The args parameter *is* passed by value!

This is not always the case. x86_64 defines it as an array of length 1,
which is passed by reference (I learned it the hard way)

[Tim Rentsch]

Gareth Owen <gwo...@gmail.com> writes:

> jacobnavia <ja...@jacob.remcomp.fr> writes:
>

>> Le 19/07/2017 at 21:00, Scott Lurndal a ecrit :

>>
>>> jacobnavia <ja...@jacob.remcomp.fr> writes:
>>>
>>>> This code
>>>> void SetErrorN(ErrorCode ec, int mc,
>>>> const char* emf = NULL, va_list args = NULL);
>>>
>>> It is not legal to set a default argument value for va_list,
>>> which is an opaque implementation defined type.
>>
>> It may be not legal now, but it compiled without any problem until
>> 2016 for at least a dozen of years.
>
> That's how opaque types work. If you guess their type correctly for a
> particular compiler, the compiler won't complain, but its not portable
> (as you've discovered). As you well know, in C there's a big difference
> between something that compiles & works in C or C++, and something that
> is actually defined as legal.

The type va_list is not an opaque type. It's not a fixed type,
but it is (required to be) a complete object type. What you mean
is that the definition of va_list is implementation dependent
(and thus the '= NULL' default might or might not work, according
to what the type of va_list is).

The problem is, as Manfred points out, although va_list must be
a complete object type, it may be an array type. That means
using a default constructor isn't portable either:

// Don't do this!
void SetErrorN(

ErrorCode ec, int mc, const char* emf = NULL,

va_list args = va_list() // NO GOOD!
);

Instead, a default-valued va_list parameter can be done portably
like this:

extern va_list empty_va_list; // must be defined somewhere

void SetErrorN(

ErrorCode ec, int mc, const char* emf = NULL,

va_list args = empty_va_list // works portably
);

If va_list is not an array type, the default value is copied; if
va_list is an array type, the parameter value is the address of
the global symbol 'empty_va_list'. This pattern works for both
kinds of implementations.

Now though there is another problem. Suppose we want to check
the 'args' parameter to see if it's an empty va_list. The
possible array-ness of the va_list type means 'args' might be a
pointer, which is hard to reconcile (ie, in portable code) with
the case where va_list is not an array type. Fortunately there
is a nice way out of this problem, which is to use a reference
type (the default argument expression should also appear in the
function's declaration, if any):

extern va_list empty_va_list; // must be defined somewhere

void SetErrorN(

ErrorCode ec, int mc, const char* emf = NULL,

va_list& args = empty_va_list // works portably
){
if( &args == &empty_va_list ){
// code here for call with no 'args' argument given
} else {
// code here for call with 'args' argument given
}
}

Now that I think about it this pattern could be useful with other
types too (with a 'const' added if need be for value arguments).
In any case one of the last two techniques should be easy to
retrofit into Jacob's environments.

[jacobnavia]

There are two issues here:

1: Try to distinguish between args that have the default value and args
that were passed by the calling function.
2: How the actual variable arguments stuff is passed in the x86-64.

To be more clearter about issue 2; I have implemented that in the x86-64
in the compiler system lcc-win.

Integer args are passed in the integer regs, floating point is passed in
the xmm registers.

So you need TWO pointers to each position within the integer registers
array and to the position in the xmm array.

Microsoft decided to use just 4 of each register set, and arguments are
always counted, so you just need one pointer. If you want to read
integer/pointer data, you index the int regs array with 4 positions, if
floating point xmm0 to xmm3

Gcc however devised a completely different and much more complicated
layout. You need two independent counters, so you can pass really a LOT
of stuff in registers instead of memory.

I am working in an arm64 and there it is even worst, with MUCH more
regs. But essentially it is the same: two regs array and the stack as a
backup.

In all cases it is a composite object.

Of course if there are more arguments than registers you still pass the
rest on the stack, so you have to keep track of that discontinuity too.

I generate all that handling inline in asm. And what was a banal bug
easy to fix with just calling the constructor of the stuff (we are in
c++ after all) started a storm of false suppositions, leading me to
believe that c++ has something completely different.

[jacobnavia]

Le 23/07/2017 à 20:27, Tim Rentsch a écrit :
> Now though there is another problem. Suppose we want to check
> the 'args' parameter to see if it's an empty va_list. The
> possible array-ness of the va_list type means 'args' might be a
> pointer, which is hard to reconcile (ie, in portable code) with
> the case where va_list is not an array type. Fortunately there
> is a nice way out of this problem, which is to use a reference
> type (the default argument expression should also appear in the
> function's declaration, if any):
>
> extern va_list empty_va_list; // must be defined somewhere
>
> void SetErrorN(
> ErrorCode ec, int mc, const char* emf = NULL,
> va_list& args = empty_va_list // works portably
> ){
> if( &args == &empty_va_list ){
> // code here for call with no 'args' argument given
> } else {
> // code here for call with 'args' argument given
> }
> }
>
> Now that I think about it this pattern could be useful with other
> types too (with a 'const' added if need be for value arguments).
> In any case one of the last two techniques should be easy to
> retrofit into Jacob's environments.

Wouldn't it be better if we kept NULL and overload comparisons of
va_list with NULL?

That would be much clearer.

NULL as a default value should be OK with nicely designed classes isn't it?

I would prefer to read
if (args == NULL)
rather than
if (&args == &empty_list)

Where we have to create an artificial object to replace NULL, and figure
out a common name for this object. "empty_va_list" is clear, but other
people could name it "no_args", and other va_list_default, or... you see?

NULL is already there for that purpose, just let's keep using that.

With operator overloading we can pass the overhead to compile time and
the code will work in any implementation, with 32 bit pointers or with
64 bit register arrays.

It shouldn't be implementation defined but language defined. We have a
list of arguments of maybe different types and sizes, and a decoder
argument, where the types are described in the context of the va_args
function.

This arguments are numbered by an integer index (first, second, etc). We
start, read the next one, etc, within the va_args function. Each next
operation returns a copy of the value (args are passed by value, i.e. by
copying).

[Manfred]

On 07/23/2017 10:30 PM, jacobnavia wrote:
> Le 20/07/2017 à 16:52, Manfred a écrit :
>> On 7/19/2017 10:44 PM, Ian Collins wrote:
>>> The args parameter *is* passed by value!
>> This is not always the case. x86_64 defines it as an array of length
>> 1, which is passed by reference (I learned it the hard way)
>
> There are two issues here:
>
> 1: Try to distinguish between args that have the default value and args
> that were passed by the calling function.
> 2: How the actual variable arguments stuff is passed in the x86-64.

I am not sure how much I can help, but I am trying to start with a
couple of preliminary considerations here.
- va_list in C++ is no better, no worst than it is in C. It has been
inherited as-is by Bjarne as part of C compatibility. Its usage is also
the same as in C (I think the constructor does not really add much, as
long as you can't have additional operations)
The official way introduced to handle variable arguments in C++ is
variadic templates. This is definitely more robust than va_list, but
unfortunately it is not compatible at all with it (i.e. it requires
substantial code rewriting).

- The way I see it, va_list is meant to be used as an opaque type, only
to be accessed via va_start, va_arg, va_end and va_copy (where
available) - I mean, other than that you give up with portability, I
believe.

As far as I can understand, the problem you are having right now arises
for not having handled va_list as opaque in the first place, and so
changing achitecture resulted in the current problem.
This is to say that, possibly, one hint would be to try to solve the
portability problem where it arises, i.e. get rid of the = default

void SetErrorN(ErrorCode ec, int mc,

const char* emf = NULL, va_list args /* = NULL */);

This breaks the code as it is now, but it makes it possible to identify
where such default is used, and replace it instead with a properly
initialized va_list - via va_start. I don't know how much feasible this
is in your case.
You would still be left with the doubt whether args is empty of not, but
this is in fact how va_list works as per standard, correct me if I am
wrong (it is one of its limitations). In C (as well as in C++) it is the
responsibility of the programmer to ensure that the callee does not
exceed the amount of arguments provided.
If this was not done in the beginning, it could be an additional problem.

If this is not possible, the second alternative would probably be using
va_list*, and use address comparison as identity check, or you can use a
NULL va_list* when empty.
cppreference.com says: "It is legal to pass a pointer to a va_list
object to another function and then use that object after the function
returns."
http://en.cppreference.com/w/cpp/utility/variadic/va_list

I would not go down the path of issue 2. It looks that you have
considerably more experience than I have, but even if you solve entirely
issue 2, you get to a solution that is not portable, so the problem is
likely to show up again with the next architecture.

Hope this helps..

[Alf P. Steinbach]

On 23.07.2017 20:27, Tim Rentsch wrote:
> [snip]

> The problem is, as Manfred points out, although va_list must be
> a complete object type, it may be an array type. That means
> using a default constructor isn't portable either:
>
> // Don't do this!
> void SetErrorN(
> ErrorCode ec, int mc, const char* emf = NULL,
> va_list args = va_list() // NO GOOD!
> );
>
> Instead, a default-valued va_list parameter can be done portably
> like this:
>
> extern va_list empty_va_list; // must be defined somewhere
>
> void SetErrorN(
> ErrorCode ec, int mc, const char* emf = NULL,
> va_list args = empty_va_list // works portably
> );

void foo( A args = A{} )
{}


- Alf

[David Brown]

For x86-64, gcc did not "devise" anything. On Windows, gcc uses the
standard Win64 calling convention just like most other compilers (and I
strongly recommend you do the same). On Linux, gcc uses the standard
*nix x86-64 calling convention (gcc was one of several parties involved
in making that calling convention - but AMD was in charge). Again, I
would strongly recommend you stick to that convention for x86-64 on Linux.

On 32-bit Windows, calling conventions were a mess - nothing was
standardised, and every tool had a different favourite and half a dozen
tool-specific qualifiers for accessing "foreign" functions. On 64-bit
Windows there /is/ a standard calling convention, and you should stick
to it. (It happens to be a bit inefficient for most function calls,
compared to the method used on *nix, but it has the advantage here of
being particularly easy for va_args functions.)

[Manfred]

On 7/24/2017 12:12 AM, Manfred wrote:
> If this is not possible, the second alternative would probably be using
> va_list*, and use address comparison as identity check, or you can use a
> NULL va_list* when empty.

By the way, for correct attribution, Ian Collins first mentioned in this
thread va_list* and variadic templates:

> Pass a va_list* instead? Or use a variadic template?

Also, NULL would be nullptr in C++

[Manfred]

On 7/23/2017 10:58 PM, jacobnavia wrote:
>
> Wouldn't it be better if we kept NULL and overload comparisons of
> va_list with NULL?
>
> That would be much clearer.
>
> NULL as a default value should be OK with nicely designed classes isn't it?

The problem is that as soon as you access the type va_list, with any
operation, and even if you manage to do that, your code becomes
non-portable.

>
> I would prefer to read
> if (args == NULL)
> rather than
> if (&args == &empty_list)
>
> Where we have to create an artificial object to replace NULL, and figure
> out a common name for this object. "empty_va_list" is clear, but other
> people could name it "no_args", and other va_list_default, or... you see?
>
> NULL is already there for that purpose, just let's keep using that.

If you really want to use NULL (i.e. nullptr in C++), then better use
va_list* (as Ian Colling suggested earlier)
In the general picture, though, this would only be a partial solution,
because this would allow you to test for an explicit empty list (a call
with nullptr as va_list*), but you would still have no information on
count and type of arguments in the general case, because this is how
va_list works.

>
> With operator overloading we can pass the overhead to compile time and
> the code will work in any implementation, with 32 bit pointers or with
> 64 bit register arrays.
>
> It shouldn't be implementation defined but language defined. We have a
> list of arguments of maybe different types and sizes, and a decoder
> argument, where the types are described in the context of the va_args
> function.
>
> This arguments are numbered by an integer index (first, second, etc). We
> start, read the next one, etc, within the va_args function. Each next
> operation returns a copy of the value (args are passed by value, i.e. by
> copying).

What you are describing here is in fact what variadic templates are for.
However, va_list is not such a thing.

[Tim Rentsch]

jacobnavia <ja...@jacob.remcomp.fr> writes:

> Le 23/07/2017 at 20:27, Tim Rentsch a ecrit :

The problem is NULL doesn't work portably for the va_list type,
because of the possibility that it can be a non-array type. On
some platforms there are good reasons to define va_list as an
array type, and on other platforms reasons to define va_list as a
non-array type. However we could define a macro NULL_VA_LIST so
the declaration could read

void SetErrorN(
ErrorCode ec, int mc, const char* emf = NULL,

va_list& args = NULL_VA_LIST
);

which IMO is sufficiently descriptive (and perhaps a little
better than using a regular lower-case variable name).

Note btw that I say "macro", but it could be defined as a
regular variable

extern va_list NULL_VA_LIST;

without clients needing to know the difference.

> I would prefer to read
> if (args == NULL)
> rather than
> if (&args == &empty_list)

How about

if ( is_null_va_list(args) ) ...

which can be provided by a static inline function

static inline bool
is_null_va_list( va_list &list ){
return &list == &NULL_VA_LIST;
}

Of course I am expecting that NULL_VA_LIST and is_null_va_list()
will both be defined in a single header file, so anyone who wants
to use default va_list parameters would #include that one header.

> Where we have to create an artificial object to replace NULL, and
> figure out a common name for this object. "empty_va_list" is clear,
> but other people could name it "no_args", and other va_list_default,
> or... you see?

If it were me I would simply go ahead with NULL_VA_LIST. It
isn't perfect, but I think it's good enough. And using a
reference type for the parameters means clients won't need
to have their source code changed.

[Tim Rentsch]

Interesting. Surprising in a way that {}'s work but ()'s don't.
I'm still digesting the differences between the two forms. Thank
you.

However using this form of initializer does not (AFAICT) allow a
means of testing the parameter to see if it got a default value.
Using a reference and 'empty_va_list' can do that.