[TCLCORE] There is no justification for modifying [proc] for named parameters

[Colin McCormack]

I don't think modifying [proc] to support a particular protocol for names args is a good idea.

I think the protocol [proc] currently has for args is sufficient and I don't think any extension of it is warranted, nor required by the Flight Aware Bounties [FABs].

I think that the challenges outlined by FABs would be amply met by defining a different command FAProc in C and packing whatever functionality is required into FAProc.

I think that the notion that [proc] needs to be extended (at a runtime cost and at documentation and implementation complexity cost) to cover a particular calling protocol is really a co-opting of the [proc] name, and not an addition of functionality.

I think that the remedy for "but I want *all* my proc to be FAProc" is [rename].  I am not aware of what, in the FABs, actually require proc to be pre-[renamed] as it were.

I am not aware of what benefit there is in adding processing cost to traditional proc, either for the users of FAProc or the users of proc.

I will now closely paraphrase the FABs to try to understand why redefining the verb proc might gain FlightAware (or anyone) anything substantive.

I paraphrase the FABs for clarity:

"A first class, high-performance, non-hackish way to do named parameters" which "must be first class, i.e. can't be a proc that wraps procs with a bunch of scaffolding" and a Named Parameter Protocol (example FAProc declaration given) which would 'replace' (traditional proc declaration given) and which is to be (and here's the kicker) "implemented as a first class thing in the Tcl code" and not to be implemented as "some proc that generates a proc or something lame like that"

The bounty is attracted to a "first class thing" which implements a names parameter protocol, which is not itself implemented as a call to proc 'or something lame like that' ... vague much?

Analysis:  any "first class thing" could be either a command or some kind of modification to the parser which applied only to calls to the proc command.  I'm going to assume the former, because the latter seems particularly poorly conceived.

It seems to me that the 'replace' part of the bounty specification is amply satisfied by [replace proc AFProc] and that the AFProc command, if implemented in C or some other method which is not "lame" would satisfy the bounty requirements.

The requirements of the bounty seem to have been reinterpreted somehow as an imperative to modify the implementation of proc.  They are not.

This re-implementation will by its nature drag on proc, will by its nature slow proc definitions which don't use the extended facilities specified by AFProc. This overhead would arise and be imposed for no good reason, but simply because the bounty was poorly phrased.

Unless FA get to pay $10k to impose new overhead on every proc call ever, for no concomitant benefit, I think the idea of modifying [proc] to save FA (et al) the time it takes to run [rename] is a *very* bad idea.  I should state, clearly, that I don't think FA get to pay $10k to impose new overhead, nor that FA want that, nor that they would obtain any benefit at all from doing that ... I think they just got carried away with how implementing language extensions in Tcl is 'lame' (which I doubt) and forgot to properly specify what they mean by "first class thing."

I have no personal desire to prevent FA from getting their "first class Tcl thing" which isn't "lame like a proc", but I really don't feel any impetus to impose a FA performance load on every [proc] call so their requirements might be satisfied in what seems to be the lamest way possible.

Colin

[Mathieu Lafon]

Hello Colin,

On Sat, May 20, 2017 at 5:11 AM, Colin McCormack <mcc...@gmail.com> wrote:
> This re-implementation will by its nature drag on proc, will by its nature
> slow proc definitions which don't use the extended facilities specified by
> AFProc. This overhead would arise and be imposed for no good reason, but
> simply because the bounty was poorly phrased.

You seems to assume that the implementation add an overhead on proc
execution, especially for proc not using the new argument specifiers,
but this is *not* the case.

Please re-read the current TIP, look at the implementation or do your
own testing, there is no performance issue.

I have carefully ensured that proc, which are not using new argument
specifiers, are not impacted in any way. On the opposite, users which
replace their own Tcl-pure code handling named parameter by using the
new specifiers will have a performance gain.

I am perfectly aware that some people are against this proposal. I
just hope that they do not have false or outdated assumptions on the
subject, as many parts have changed since the initial proposal.

That's why I'm currently requesting a vote, so that I can have a clear
position of TCT members on whether this can go into core or whether
this should be handled completely differently.

Deciding whether this should be handled in proc or handled differently
(new command, calling an extension inside proc, ...) is not something
on which we can reach a consensus (at least I have clearly failed to
achieve that). A decision has to be done.

Regards.

-- Mathieu

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Tcl-Core mailing list
Tcl-...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/tcl-core

[Colin McCormack]

Hi Mathieu,

I don't believe you.  Anything which adds functionality to an optimally implemented proc must add overhead to it.  You might argue that proc as implemented is sub-optimal, in which case thank you for finding a way to optimise it.  You might argue that you cannot measure any difference between the optimal and the extended versions, in which case you need to measure more closely.  You might argue that the difference is insignificant, or that it's worth the cost, which begs the question of why using proc to provide the extension is significant or not worth the cost.

If you're honestly arguing that you have two (otherwise identical) pieces of code, one of which implements a discriminator and one of which doesn't and the one which performs more computation does so with no additional computation time, then either I have a fundamentally flawed view of what computation is, or you do.

I simply cannot buy it.

Colin.

[Colin McCormack]

Erratum: clearly when I say 'computation time' when speaking of overhead, I should have said 'computation time or space.'

[Alexandre Ferrieux]

On Sat, May 20, 2017 at 10:38 AM, Mathieu Lafon <mla...@gmail.com> wrote:
> That's why I'm currently requesting a vote, so that I can have a clear
> position of TCT members on whether this can go into core or whether
> this should be handled completely differently.

Though you already know my position (close to Colin's), I'll take this
opportunity to clarify the counter-proposal:

- it clearly must be in C (thus reuse 99% of your impressive work)

- it could either be proc-like (like Colin's FAproc) or a simpler
runtime handler ([eatargs $args] or even [eatargs] since [info level
0] introspection allows for it). My preference goes to the second
since it's simpler / less tentacular and works well with oo methods
and apply.

- it would prefer, as a secondary goal, to have it as a bundled
extension like Thread and Tk rather than in the core, for a probation
period. Reason is twofold: (1) there's room for months of refinement
of the script-level API ; (2) its ubiquity might complicate
Tcl-to-other-lang translation. By "secondary goal" I mean I would
still vote YES (after reasonable arm twisting) if only the first two
goals were satisfied. Otherwise, as it stands, it is NO.

-Alex

[Colin McCormack]

I think the [eatargs] thing is fine, and clearly FAProc could trivially be implemented in terms of it.

A real show-stopper for me with the "let's extend proc" approach (if I had any actual say in the matter of what Tcl as implemented becomes, which I don't) is that [proc] as [FAProc] becomes radically dissimilar to all the other command-generating commands [CGCs].

If the fundamental script-level CGC is no longer able to simulate, or be simulated by, the others then Tcl loses enormously.

Colin

[Mathieu Lafon]

I have not said that there are no overhead in using the new extended
specifiers, I have said that there are no overhead if the proc is not
using them.

A proc defined with extended specifiers is flagged so that it uses a
different code path for locals initialisation. The original and
optimized code path is still used for proc which are not flagged.

If we're discussing the impact of the added flag check in the code, I
admit that there is an impact on the generated compiled code but I
will not call it an overhead.

-- Mathieu

[Colin McCormack]

Well, of course it's an overhead, by definition.  Are you arguing that it's insignificant, or that it's cost effective?

On, now, to the next element of overhead, which you haven't dealt with (here.  I haven't been following closely, I'm sorry if you have dealt with it elsewhere) which is the *considerably* more complex documentation.  Not significant?

Finally, and for me this really would end the matter (I think it's just that important), the loss of ability to bisimulate CGCs in Gcl.

Colin.

[Colin McCormack]

Argh, I meant "in Tcl", of course.

[Colin McCormack]

Oh, and I think I may have misused "bisimulate" ... I meant that every CGC form should be able to imitate every other so that it's impossible for the caller to tell when one CGC-command has been substituted by another of identical functionality simply by calling it.  They should all look the same.  It matters.

[Mathieu Lafon]

I don't want to enter a flame-war with you. I understand that you're
against it and I completely respect your point of view but I don't
think I will change your mind.

I admit that I have failed to reach a consensus but please respect the
work done and the opinion of the people who have supported the work.

-- Mathieu

[Colin McCormack]

Excuse me, Mathieu, but I'm not simply "against it."

I have raised what I consider to be cogent criticisms of a proposed course of action, and as far as I can tell you have addressed none of them.  I sought to understand why our opinions differ about what is best.

To denigrate my analysis as 'flaming' doesn't discredit it, and to fail to meaningfully engage with it surely only undermines your proposal.

I believe that you have misread the bounty's stated requirements, if you believe that modifying [proc] is essential to satisfying them.

Whence did the idea arise that distorting proc was the best way to add the functionality you appear to desire, or desire to implement?

Colin

[Colin McCormack]

I forgot: increased implementation complexity of proc is the other deoptimisation your proposal must obviously entail.  All the same defences are available to you as for performance degredation.

I would expect any proposal such as yours to have considered and addressed these issues.  I didn't expect that countering my critique would be onerous if you had already considered them.

I certainly didn't think that politely raising some fairly obvious objections could be called a "flame-war."  Not with a straight face, anyway.

Colin

[Colin McCormack]

*degradation

[Kevin Kenny]

On 05/20/2017 04:53 AM, Colin McCormack wrote:
>
> Hi Mathieu,
>
> I don't believe you. Anything which adds functionality to an
> optimally implemented proc must add overhead to it. You might argue
> that proc as implemented is sub-optimal, in which case thank you for
> finding a way to optimise it. You might argue that you cannot measure
> any difference between the optimal and the extended versions, in which
> case you need to measure more closely. You might argue that the
> difference is insignificant, or that it's worth the cost, which begs
> the question of why using proc to provide the extension is significant
> or not worth the cost.
>
> If you're honestly arguing that you have two (otherwise identical)
> pieces of code, one of which implements a discriminator and one of
> which doesn't and the one which performs more computation does so with
> no additional computation time, then either I have a fundamentally
> flawed view of what computation is, or you do.
>

In an ideal implementation of this, the overhead, for procs that do not
use the new functionality, will be all in proc definition, not proc
execution.

Since proc definition is a vanishingly rare event compared with churning
away in inner loops, I expect that for typical scripts the overhead will
not be measurable.

It would appear that Mathieu thought that you were speaking of
execution-time overhead.

[Mathieu Lafon]

Colin,

My intent was not to denigrate your analysis but to try to calm things
down. Sorry for having triggered the opposite...

We have a disagreement on how this should have been implemented. I
have chosen to modify proc argument specifiers after having proposed
other options (including a dedicated command to call a proc :
http://www.tcl.tk/cgi-bin/tct/tip/view/457?ver=1.7) and received
feedback in that way. The initial FA bounty description was requiring
that it must work with unmodified proc, so the spirit of the bounty
seems to favor keeping proc and not adding a separate command.

Having chosen to modify proc, this implies impact on code, performance
and documentation, which I have tried to keep minimal, but are
inherent to that decision. I have carefully ensured that the new code
is well separated so that it is easy to review and maintain and that
the proc not using extended specifiers are not impacted in any way.

Regarding the two proposals made by Alexandre :

- Adding a new FAproc command will require to duplicate a lot of code
already present in proc, leading to more maintenance. Users may find
confusing to have a separate command and new users may not find it.

- Adding a runtime handler seems to be limited to argument parsing,
where the current proposal is a base framework which can be easily
extended to later support other usecase on proc arguments (type
assertion, doc-string, ...). Unless I don't get the intent, it seems
necessary to specify the named arguments when calling the command.
Users may find counter-intuitive to have to specify the arguments in a
second time (I assume proc only use 'args' in that case).

I don't have any strong positions on any alternate proposal, I only
want to give my current proposal a chance to be considered.

-- Mathieu

On Sat, May 20, 2017 at 12:41 PM, Colin McCormack <mcc...@gmail.com> wrote:
> Excuse me, Mathieu, but I'm not simply "against it."
>
> I have raised what I consider to be cogent criticisms of a proposed course
> of action, and as far as I can tell you have addressed none of them. I
> sought to understand why our opinions differ about what is best.
>
> To denigrate my analysis as 'flaming' doesn't discredit it, and to fail to
> meaningfully engage with it surely only undermines your proposal.
>
> I believe that you have misread the bounty's stated requirements, if you
> believe that modifying [proc] is essential to satisfying them.
>
> Whence did the idea arise that distorting proc was the best way to add the
> functionality you appear to desire, or desire to implement?
>
> Colin
>
>
> On Sat, 20 May 2017, 20:02 Mathieu Lafon, <mla...@gmail.com> wrote:
>>
>> I don't want to enter a flame-war with you. I understand that you're
>> against it and I completely respect your point of view but I don't
>> think I will change your mind.
>>
>> I admit that I have failed to reach a consensus but please respect the
>> work done and the opinion of the people who have supported the work.
>>
>> -- Mathieu

[Colin McCormack]

On Sat, 20 May 2017, 23:32 Kevin Kenny, <kevin....@gmail.com> wrote:

In an ideal implementation of this, the overhead, for procs that do not
use the new functionality, will be all in proc definition, not proc
execution.

I would hope that's the case here.

Since proc definition is a vanishingly rare event compared with churning
away in inner loops, I expect that for typical scripts the overhead will
not be measurable.

I agree with you that for many (probably most) scripts the definition of procs is vanishingly rare.  It follows that any reasonable overhead in the service of a desired facility in defining a command is probably warranted.

This argument is a more effective argument in favour of writing FAProc in Tcl than it is to extend proc.  That someone finds such an implementation 'lame' is no argument at all for modifying proc.

It would appear that Mathieu thought that you were speaking of
execution-time overhead.

I see.  Thank you for interpreting his misunderstanding to me.

Colin

[Colin McCormack]

On Sat, 20 May 2017, 23:36 Mathieu Lafon, <mla...@gmail.com> wrote:

Colin,

My intent was not to denigrate your analysis but to try to calm things
down. Sorry for having triggered the opposite...

We have a disagreement on how this should have been implemented. I
have chosen to modify proc argument specifiers after having proposed
other options (including a dedicated command to call a proc :
http://www.tcl.tk/cgi-bin/tct/tip/view/457?ver=1.7) and received
feedback in that way. The initial FA bounty description was requiring
that it must work with unmodified proc, so the spirit of the bounty
seems to favor keeping proc and not adding a separate command.

I am blissfully unaware of the orifinal terms of the bounty, but as I read it now there is absolutely no requirement that proc be modified.

I am also really dismayed that one might be able to bid for mods to core commands.

Having chosen to modify proc, this implies impact on code, performance
and documentation, which I have tried to keep minimal, but are
inherent to that decision. I have carefully ensured that the new code
is well separated so that it is easy to review and maintain and that
the proc not using extended specifiers are not impacted in any way.

I have no reason to cast aspersions on your code.  I merely seek to criticise your quite arbitrary choice to modify proc to insert functionality like this.

Regarding the two proposals made by Alexandre :

- Adding a new FAproc command will require to duplicate a lot of code
already present in proc, leading to more maintenance. Users may find
confusing to have a separate command and new users may not find it.

I don't believe code is ever required to be duplicated.  Refactoring proc so as to expose an interface which FAproc can use would always be an option.

- Adding a runtime handler seems to be limited to argument parsing,
where the current proposal is a base framework which can be easily
extended to later support other usecase on proc arguments (type
assertion, doc-string, ...).

Since the proposal was, I believe, to implement named arguments, I can well understand why one would expect it to be limited to argument handling.

Unless I don't get the intent, it seems
necessary to specify the named arguments when calling the command.
Users may find counter-intuitive to have to specify the arguments in a
second time (I assume proc only use 'args' in that case).

It sould be possible (I think trivial) to use such a handler to implement FAProc in all its glory and wonder, however it's conceived or the concept evolves.

I don't have any strong positions on any alternate proposal, I only
want to give my current proposal a chance to be considered.

Well, I considered it, and I think changing proc so its resultant command no longer resembles those of apply, namespace ensemble, interp, coroutine etc is a great mistake.

Colin

[Andreas Leitgeb]

Colin McCormack <mcc...@gmail.com> wrote (mix of some mail)

> I am also really dismayed that one might be able to bid for mods to core
> commands.

The bid may have been motivation for mlafon to spend his time to fine-design
and implement it, but it has not more impact on TCT's decision, than a mere
suggestion of someone. IOW, you can bid someone to implement something, but
you cannot "bid something into the core".

> I have no reason to cast aspersions on your code. I merely seek to
> criticise your quite arbitrary choice to modify proc to insert
> functionality like this.

This looks really strange, Colin: you write you wouldn't want it in the
core "for it would then have proc interfaces be different from apply's
or OO-method's." The opposite is true: if the named arguments were done
as a separare faproc, *then* faproc-generated commands would be singly
more powerful than plain proc/apply/methods.

The current implementation is done at a level so that apply and oo-methods
also benefit for free from it, just like proc, so most (if not all) of
these CGC would still be equivalent! And then they're also a step closer
towards equivalence to what standard compiled commands do.

That the TIP also covers apply and oo-methods has been repeatedly stated
here...


Another more fundamental aspect of TIPs in general:
A TIP is a proposal, with the author's intention to "improve Tcl".

Apparently, there is not always a consensus about whether something *is*
an improvement or not, but it makes no sense at all to say: "but you
could do some of it in <this-roundabout-way> or <that-roundabout-way>."
The very point of a TIP is, that the respective author is *not* fine
with the workarounds, and *therefore* suggests a change to *improve*
Tcl towards *not* needing the workarounds.

Without "improvements" we still wouldn't have $var, {*}-expansion,
and maybe "proc" itself would be an extension to download and compile
separately...


> I don't believe ...

and discussions here would be more compact, if some of these beliefs or
disbeliefs were checked against facts before being thrown in as arguments.

Even if FA's bounty *could* be met with a different solution, then this
TIP proposes a particular solution, and this solution shall be judged,
or an alternative proposal made (if the objective is agreed but only the
approach disapproved).

That "FA demands introspectability" is not the only point against some
parseargs-approach. The point is rather that introspection *is* a worthy
concept to care for. And FA just added it as a condition, as they appa-
rently didn't fancy a half-a..ed solution - neither would I.

[Colin McCormack]

I see now why mlafon was anxious about a flamefest arising - enter the co-sponsor, and now it's like some tag-team of issue-avoidance.

If nothing else comes out of this discussion, I would hope it's the distinction between a command improvement (which Andreas et al argues this is) and a Tcl improvement (which I argue it is not.)

Responses intercut below.

On Sun, 21 May 2017, 01:47 Andreas Leitgeb, <a...@logic.at> wrote:

Colin McCormack <mcc...@gmail.com> wrote (mix of some mail)
> I am also really dismayed that one might be able to bid for mods to core
> commands.

The bid may have been motivation for mlafon to spend his time to fine-design and implement it, but it has not more impact on TCT's decision, than a mere suggestion of someone. IOW, you can bid someone to implement something, but you cannot "bid something into the core".

It was a throw-away line on my part, but I think it's worth saying.  There's no allegation of corruption, merely a recognition that the TCT votes on TIPs, but the specifier of TIPs specifies mods.

A major problem of this TIP is that it exploits the fact that the ::proc argument protocol [PAP] is very loose, by restricting it and imposing consequential and potential incompatibility on *every* PAP forever after.

To improve Tcl, this TIP doesn't have to be merely an Improvement to PAP as implemented by ::proc, it has to be the best *possible* improvement to PAP, because it modifies a fundamental command and thereby forecloses on a presumably infinite number of possible future PAPs.

That's a heavy responsibility which the TIP doesn't seek to rise to.

The first TIP that restricts any fundamental Tcl command like this is also the last one.

By paying to restrict ::proc, one selects which possible ::proc-restriction will drive out any other possible proc restriction.  In this case, one bids a modification into the core which is *not* materially entailed by the stated specified requirements.  It's a land-grab, where the real estate occupied is the name ::proc.

> I have no reason to cast aspersions on your code.  I merely seek to
> criticise your quite arbitrary choice to modify proc to insert
> functionality like this.

This looks really strange, Colin: you write you wouldn't want it in the
core "for it would then have proc interfaces be different from apply's
or OO-method's."  The opposite is true: if the named arguments were done as a separare faproc, *then* faproc-generated commands would be singly more powerful than plain proc/apply/methods.

I think Alexandre's counter-proposal for a generic arg-handler at runtime is better for precisely that reason.  Any user of the other CGCs  could elect to follow FA's arg protocol [FAP] if the author wished.

I think that, if given the option to include FAP, most authors probably wouldn't, but this TIP takes away even that choice.

The current implementation is done at a level so that apply and oo-methods also benefit for free from it, just like proc, so most (if not all) of
these CGC would still be equivalent!  And then they're also a step closer
towards equivalence to what standard compiled commands do.

Oh!  So the TIP modifies *all* CGCs?  I was unaware of that, from reading it.  That's troubling. 

That the TIP also covers apply and oo-methods has been repeatedly stated
here...

I read the TIP just then, I didn't see that stated.  What of all the other CGCs?  Is your argument against fragmenting the arg protocols in the manner I mention seriously that you're proposing to change a larger subset of them than I'd realised?

Another more fundamental aspect of TIPs in general:
A TIP is a proposal, with the author's intention to "improve Tcl".

Apparently, there is not always a consensus about whether something *is* an improvement or not,  but it makes no sense at all to say: "but you
could do some of it in <this-roundabout-way> or <that-roundabout-way>."
The very point of a TIP is, that the respective author is *not* fine
with the workarounds, and *therefore* suggests a change to *improve*
Tcl towards *not* needing the workarounds.

And of course the problem with any improvement is that if it forecloses other changes it can be a local maximum.  Instead of proposing a new CGC with a new name which has extra options, this proposal coopts the name ::proc to include those options.

Is this the best of all possible ::procs?  I doubt it, but I don't know, and I see no evidence in the TIP that the authors have considered that weighty question.

That a given author has elected to implement their proposed command improvement in a certain way is really no guarantee that the improvement couldn't be improved.

I see absolutely no argument even attempted that the arbitrary choice to modify ::proc in (a way which practically cannot be reversed) is necessary, or desirable, or minimal or an improvement to Tcl (although any increase in functionality of a given command can be trivially be construed as an improvement to that command.)

The *only* justification I have seen for the choice to mod ::proc, rather than to create a new CGC is that FA paid for just that (and, incidentally, I see no evidence for that in the bounty specification.)

Such a major change to a fundamental command in Tcl isn't so much a design choice, in my opinion, as a fundamental mistake.

Without "improvements" we still wouldn't have $var, {*}-expansion,
and maybe "proc" itself would be an extension to download and compile
separately...

Those were genuine improvements, and *much* more consideration was given to their Tcl-wide effects than has been given to this.

Moreover: {*} went through rapid revision from {expand}, demonstrating that its authors were flexible enough to reconsider their initial design choices when presented with better ones (see Alexandre's counter-proposal.)

If I were you, I wouldn't cite $var in support of your radical language mod, because it too easily leads to a discussion about $() and WTF is between those parentheses.

> I don't believe ...

and discussions here would be more compact, if some of these beliefs or
disbeliefs were checked against facts before being thrown in as arguments.

That's almost a tautology, no?

I don't know that 'compact discussion' is really the highest good when addressing radical language modification, lest it turn out to be mutilation, unless of course one stands to directly benefit from speedy resolution.

Even if FA's bounty *could* be met with a different solution, then this
TIP proposes a particular solution, and this solution shall be judged,
or an alternative proposal made (if the objective is agreed but only the
approach disapproved).

As an aside, I would like to have seen "the objective" defined more clearly and with less of the "or something lame like that" verbiage, which seems like dog whistling, and which isn't susceptible to rational analysis (but I repeat myself.)

If you're really maintaining that the objective is specified, you might be kind enough to direct me to where the objective of modifying ::proc is specified as a requirement, and where it's justified over the available alternatives.

That "FA demands introspectability" is not the only point against some
parseargs-approach. The point is rather that introspection *is* a worthy
concept to care for. And FA just added it as a condition, as they appa-
rently didn't fancy a half-a..ed solution - neither would I.

I missed that specification/demand.  Perhaps you could point me to it.

Colin

[Colin McCormack]

I note, Kevin, from recent reading of the TIP, that the proposal seems to propose to slow $args processing, which is an execution-overhead.

Since, despite protestations to the contrary, execution-time is impacted, I wouldn't regard "But I couldn't find a way to measure it" reassuring.

If, and as, each user of $args will pay a performance penalty, it seems at least polite to tell them what it will be.

Colin.

[Alexandre Ferrieux]

On Sat, May 20, 2017 at 4:06 PM, Mathieu Lafon <mla...@gmail.com> wrote:
> Regarding the two proposals made by Alexandre :
>
> - Adding a new FAproc command will require to duplicate a lot of code
> already present in proc, leading to more maintenance. Users may find
> confusing to have a separate command and new users may not find it.

I concur with Colin's answer to this: appropriate refactoring may
avoid any duplication. But never mind, as stated, this is not my
preferred choice.

> - Adding a runtime handler seems to be limited to argument parsing,
> where the current proposal is a base framework which can be easily
> extended to later support other usecase on proc arguments (type
> assertion, doc-string, ...).

Yes, and to me this lack of ambition is a bonus. Think KISS principle.
Do one thing at a time, and do it right. Don't open a Pandora box on
vague promises.
In this specific case: type assertions and doc strings can quite
easily fit in handlers too, or in formatted comments at the top of the
proc's body.

> Unless I don't get the intent, it seems
> necessary to specify the named arguments when calling the command.
> Users may find counter-intuitive to have to specify the arguments in a
> second time (I assume proc only use 'args' in that case).

Please give an example, I'm lost. You may even give a hefty list of
them, as an exercise for me to map to [eatargs] form :)

-Alex

[Colin McCormack]

I really think your counter-proposal is best, Alexandre.  It achieves the stated goal, it trivially permits a parallel ::proc (I have been calling ::FAProc) and it imposes *nothing* on any part of Tcl.  It genuinely improves Tcl, rather than putatively improving a single command.

I have repeatedly asked what justification exists for modifying ::proc over the ::FAProc or ::eatargs counter-proposals.  None is forthcoming.

In my dark moments I fear that the reason the alternatives are resisted is precisely because they renders this command putative-improvement optional, and on some level its proponents understand that there are a slew of packages available to do what they're proposing, and people don't use them ... and probably not because those implementations have substantial overheads.

In my more forgiving moments, I consider it's just possible (particularly in terms of the "like it or lump it" responses to suggestions) that the authors have just fallen into myopic value-rigidity, having written something at considerable expense to ape and extend ::proc, they are wedded to the particular implementation choices they've made.  That's understandable, it's human, I do it all the time.

There's another possibility, I don't know what to think about it, that the ::proc-mod has been commissioned - that it's a stated/explicit or even implicit/dog-whistled requirement of the FA bounty.  I really don't know what to think about that, if so.

Anyway, absent a justification, I can't help but speculate about what seems to be an irrational decision.  I hope, though, my objections to it are reasonable and not equally irrational.

Colin

[Colin McCormack]

I'm doing a bit of language-lawyering over https://github.com/flightaware/Tcl-bounties ... which I presume is the canonical FA Bounty.  I think I see the source of the confusion about ::proc.

In item "A first class, high-performance, non-hackish way to do named parameters" the first line is "Invoke a proc by naming the variables and their values rather than doing it positionally."

Right out of the gate, it's a mess.  The problem here is that you can't "invoke a proc" in Tcl, you can only invoke a command.  I reckon there's the source of the confusion.

I'm going to suppose that they didn't mean "invoke a proc" (which isn't possible in any extant definition of ::proc) but "create a command which can be invoked by naming the variables and their values rather than doing it positionally".  That's kinda clearer.  I guess they meant, too, that the creation of a command is meant to be by way of associating a Tcl script with a command's name.  I would suppose, item "Needs to be natural and "native Tcl" like for the caller" softly implies that whatever mechanism resembles ::proc's definitional form, that it looks like ::proc ... that'd be the simplest interpretation.

Now ... I think it's most likely that the initially ill-defined requirement "invoke a proc" has been misinterpreted in the least destructive manner possible while failing to properly distinguish between "a command" and "a proc", and that this misinterpretation has been carried on and expanded in meaning to become a supposed imperative to redefine ::proc.  That's an obvious problem with reasoning from a contradiction, you can deduce anything you like.

Anyway, can we dispense with the assertion that the bounty requires/mandates ::proc modification, and conclude that it's just what the authors chose to do?  Can we, further, surmise that the misinterpretation as a requirement mislead them to consider only this approach and discount other approaches? Can I posit that as a consequence, we've ended up with an ill-considered design and a flawed proposal?

Colin

[Donald Arseneau]

OK, I'm not going to quibble about no impact versus no overhead
versus immeasurable effect.

One thing that did strike me, though, in the recent exchange was
that the new proposed proc would be harder to emulate. It is
ironic that a feature designed to avoid "scaffolding" prevents
other scaffolding. I don't think that is a show-stopper though.

I see two main reasons to wrap/rename proc:

1) Debugging tools that track proc definitions and insert
some code into each.

2) The addition of some new features for proc.

In the case of 1, I don't think there is a need to parse the
argument list for simple debugging goals, and if argument tracking
is needed then it is feasible to do so (with work). For 2, the need
to emulate maned parameters could be a lot of work, and a barrier
too difficult to overcome; but such revised commands don't need to
use the name "proc" and then wouldn't need to support named arguments.
That seems better than restricting the scope of the core "proc".

My question is about any other reasons. Is there a necessary proc-
wrapping that would be ruined by the named-arguments handling?

PS to Colin: "invoke a proc" is perfectly clear. Hint: look at
[info proc]. That comment gets you on the threshold of my *plonk*
bin.


Donald Arseneau, TRIUMF CMMS, as...@triumf.ca

[Andreas Leitgeb]

Given that Colin elaborated verbosely on his speculations about all the
tip457 business being solely bounty-triggered, and dismissing the thought
that it might be even considered a good thing, I'll counter with the
speculation that Colin is just principially against any change to Tcl
whatsoever, and he actively tries to turn this into a flame-war merely
to discredit the topic.

> It was a throw-away line on my part, but I think it's worth saying.

I copy your words for this paragraph.


Now, about CGC, PAP and the scope of TIP-457:

CGC: Code Generated Commands (iirc), seems to address all commands that
come into existence as consequence of already existing commands being
called. That would include Tk/Ttk widgets, interp aliases, namespace
ensembles, oo-classes&objects, proc and oo-methods. ("apply" isn't a
CGC, for that matter)

Obviously, most of them have no relation whatsoever to the "PAP" - the
"::proc argument protocol", which for now is shared by apply and
oo-methods, and potentially by any extension that uses Tcl_CreateProc().

If your intended meaning of CGC is a different one, then let us know what
it includes besides those that involve Tcl_CreateProc() at creation.


Tcl as a language consists of three(!) parts: the syntax, the standard
commands, and the "infrastructure". The infrastructure is what makes the
commands mostly consistent in acceptance of numbers, booleans, lists,
dicts, ...

If we were to enhance list syntax (how strings get parsed to lists), we'd
rather not add a new xlindex, xlappend, xlset, xforeach,... but instead
enhance the appropriate Tcl_...List... function.

Tcl_CreateCommand is also infrastructure. It is used by ::proc, ::apply
and oo's method and probably almost every extension that intends to wrap
some lines of tcl code to run in their own stackframe with a specified
set of local variables pre-assigned.

Now, if the PAP *were* to be enhanced, then most obviously Tcl_CreateProc
(and/or its internally called helpers) is the thing to modify.

So much for the actual motivation for TIP 457's target.

On Sat, May 20, 2017 at 11:37:36PM +0000, Colin McCormack wrote:
> A major problem of this TIP is that it exploits the fact that the ::proc
> argument protocol [PAP] is very loose, by restricting it and imposing
> consequential and potential incompatibility on *every* PAP forever after.

It is well known, that there is only one chance to ever define a meaning
for argument specifiers of length > 2. The TIP does a good job of leaving
"hooks" for further enhancements as options to be added lateron.

It may preclude other means to enhance arg-specs, but given that:
- "1 or 2 elements" is already taken,
- decorated param-name is not precluded by the tip, and
- adding even just a third element would imply presence of a default,
then it seems that any reasonable alternative wouldn't be all that
fundamentally different from 457.

I'd be curious about even rough sketches of alternatives to enhance the
PAP compatibly, that would be *precluded* by acceptance of tip 457.
This is not about laying the burden on proof on you, but it is about
pointing out mistakes in my above reasoning about possible alternatives.

> I think Alexandre's counter-proposal for a generic arg-handler at runtime
> is better for precisely that reason. Any user of the other CGCs could
> elect to follow FA's arg protocol [FAP] if the author wished.

Having non-introspectable proc foo args ... is just not the objective.

No, FA seems not to mention it - anymore? - ... no idea where I picked that
up ... anyway, as I said it wasn't for them, anyway, but because introspect-
ability *should* have been a requirement, imho.

> > The very point of a TIP is, that the respective author is *not* fine
> > with the workarounds, and *therefore* suggests a change to *improve*
> > Tcl towards *not* needing the workarounds.
> And of course the problem with any improvement is that if it forecloses
> other changes it can be a local maximum.

That's some curse of the world as it is. If you never change anything
out of fear that it could preclude something not-even-yet-fathomed later,
then it grinds to a halt ways below even the local maximum.

> Is this the best of all possible ::procs? I doubt it, but I don't know,
> and I see no evidence in the TIP that the authors have considered that
> weighty question.

The discussion has passed various stages and suggestions have been
added, refined, dropped. Almost all of my own suggestions have been
replaced by better (as I now understand) alternatives - something that
at least makes me confident that I'm not merely "defending my own stuff".

> That a given author has elected to implement their proposed command
> improvement in a certain way is really no guarantee that the improvement
> couldn't be improved.

It has been improved a lot during the discussions.
IMHO, another month or so of discussion might still improve it, in
particular for making sure that the other TIP's [info linkedname] actually
can fill the gap of dropped "-varname" (I really hope it can!)

> I see absolutely no argument even attempted that the arbitrary choice to
> modify ::proc in (a way which practically cannot be reversed) is necessary,

> or desirable, or minimal or an improvement to Tcl ...

I'm sorry to read about that physical condition of yours.

> > Without "improvements" we still wouldn't have $var, {*}-expansion,
> > and maybe "proc" itself would be an extension to download and compile
> > separately...
> Those were genuine improvements, and *much* more consideration was given to
> their Tcl-wide effects than has been given to this.

"*much* more consideration" was indeed necessary because (at least) $var
was obviously not "compatible" in the strict sense.

> Moreover: {*} went through rapid revision from {expand}, demonstrating that
> its authors were flexible enough to reconsider their initial design choices
> when presented with better ones (see Alexandre's counter-proposal.)

{*} was proposed nearly same time as {expand}. It was during discussion that
the {expand} was preferred for verbal clarity, but pragmatic rationality
fortunately came back just in time, even if only after the {expand}-vote.

Alexander's counter-proposal isn't "better", because it lacks introspecta-
bility. And i15y is obviously *not* for any FAB, but because it makes sense.

> If I were you, I wouldn't cite $var in support of your radical language
> mod, because it too easily leads to a discussion about $() and WTF is
> between those parentheses.

Despite the flaws of current $... spec-/implementation, I'm ways happy
to have it, and I've written a TIP to try to amend some of the flaws.
Offered an alternate timeline where $var wasn't added, but instead still
waiting for the perfect change, I'd choose ours.

And I'd prefer a future timeline of enhanced PAP to one of leaving PAP
as it is (in expectation of some utopic perfect enhancement).

[Mathieu Lafon]

Hello Alexandre,

On Sun, May 21, 2017 at 2:01 AM, Alexandre Ferrieux
<alexandre...@gmail.com> wrote:

> Please give an example, I'm lost. You may even give a hefty list of
> them, as an exercise for me to map to [eatargs] form :)

Let's use an example from modified proc.n. Here it is with proposed specifiers:

% proc log {
{level -name level -switch {{debug 0} {error 3}} -default 1}
{time -name {time timestamp}}
{message} } {
if {![info exists time]} {set time [clock seconds]}
array set levels {0 DEBUG 1 INFO 2 WARN 3 ERROR}
puts "[clock format $time] : $levels($level) : $message"
}

And two call examples:

% log -level 2 "Warning..."
Sun Apr 23 20:29:41 UTC 2017 : WARN : Warning...
% log -timestamp 946684800 -debug -- -Y2K-
Sat Jan 01 00:00:00 UTC 2000 : DEBUG : -Y2K-

What will it look like with [eatargs]?
- How are local variables defined? Will they be created by [eatargs] at runtime?
- How are defined the named arguments? Is it a parameter of [eatargs]
or are they defined elsewhere?

-- Mathieu

[Mathieu Lafon]

On Sun, May 21, 2017 at 1:37 AM, Colin McCormack <mcc...@gmail.com> wrote:
>> That the TIP also covers apply and oo-methods has been repeatedly stated
>> here...
>
> I read the TIP just then, I didn't see that stated.

Current version of the TIP (last edited days before this thread) has
the following dedicated section:

OTHER USE CASES
-----------------

Extended argument specifiers can also be used with other /proc/-like
functions. The following functions are supported and can use extended
argument specifiers:

* anonymous functions (lambda), used with /apply/ command ;

* TclOO constructor or methods.

On Sun, May 21, 2017 at 1:58 AM, Colin McCormack <mcc...@gmail.com> wrote:
> I note, Kevin, from recent reading of the TIP, that the proposal seems to
> propose to slow $args processing, which is an execution-overhead.

What are you talking about?

-- Mathieu

[Colin McCormack]

What I am talking about is an apparent difference of opinion as to what "covers" means ... if you had said "mentions as a possibility without actually proposing", or "glancingly refers to, before passing on," instead of "covers", I suspect we'd not have any disagreement.

I get that you've set out to provide an improved version of ::proc, and have likely done a good job of that.  I just don't think it improves Tcl.

Colin

[Colin McCormack]

On Sun, 21 May 2017, 20:28 Andreas Leitgeb, <a...@logic.at> wrote:

Given that Colin elaborated verbosely on his speculations about all the
tip457 business being solely bounty-triggered, and dismissing the thought that it might be even considered a good thing,

I certainly never said, nor implied, anything of the kind.

What I have said is that I see no justification in the tip, nor anywhere else, for the decision to make tip457 extend/modify/mutilate ::proc, rather than simlly make a parallel command.

My speculation on the bounty was purely and entirely a fruitless attempt to find *some* rationale somewhere for what is otherwise an inexplicable decision - certainly an unexplained decision.

I'll counter with the
speculation that Colin is just principially against any change to Tcl
whatsoever, and he actively tries to turn this into a flame-war merely
to discredit the topic.

Nah.  Most changes to Tcl are welcome to me.  Not this one.

> It was a throw-away line on my part, but I think it's worth saying.
I copy your words for this paragraph.


Now, about CGC, PAP and the scope of TIP-457:

CGC: Code Generated Commands (iirc), seems to address all commands that come into existence as consequence of already existing commands being called. That would include Tk/Ttk widgets, interp aliases, namespace ensembles, oo-classes&objects, proc and oo-methods. ("apply" isn't a CGC, for that matter)

Obviously, most of them have no relation whatsoever to the "PAP" - the "::proc argument protocol", which for now is shared by apply and oo-methods, and potentially by any extension that uses Tcl_CreateProc().

The important relationship I am referring to is that, as the PAP *calling* protocol comprises a list of values, any imposition of a protocol involving named parameters, call by name, etc, results in the possibility of creation of commands by ::proc which simply cannot be simulated (or cannot simply) be stimulated with other CGCs such as ::apply, ::coroutine, etc.

If your intended meaning of CGC is a different one, then let us know what
it includes besides those that involve Tcl_CreateProc() at creation.

Pretty simple, really.  You don't believe that commands can only be created by Tcl_CreateProc, you just can't imagine that there's any value in the ability to simulate those commands with others.

Tcl as a language consists of three(!) parts:  the syntax, the standard
commands, and the "infrastructure". The infrastructure is what makes the
commands mostly consistent in acceptance of numbers, booleans, lists,
dicts, ...

If we were to enhance list syntax (how strings get parsed to lists), we'd
rather not add a new xlindex, xlappend, xlset, xforeach,... but instead
enhance the appropriate Tcl_...List... function.

I don't think I would recommend 'enhancing' list parsing by modifying the [list] command, and I would strenuously counsel against doing so if your objective was to create dicts.  I would, rather, suggest that you use a different command to construct your 'enhanced' lists, such as (for a worked example) ::dict.  That's all I am suggesting here.

I have nothing against having a different way to parse scripts so as to construct commands.  I am only objecting to the completely unnecessary confusion of the old and new forms.

Tcl_CreateCommand is also infrastructure.  It is used by ::proc, ::apply
and oo's method and probably almost every extension that intends to wrap
some lines of tcl code to run in their own stackframe with a specified
set of local variables pre-assigned.

Now, if the PAP *were* to be enhanced, then most obviously Tcl_CreateProc
(and/or its internally called helpers) is the thing to modify.

So much for the actual motivation for TIP 457's target.

Perhaps my usage of "protocol" in this has been unclear.  I am using it in the same sense Occam used it.

On Sat, May 20, 2017 at 11:37:36PM +0000, Colin McCormack wrote:
> A major problem of this TIP is that it exploits the fact that the ::proc
> argument protocol [PAP] is very loose, by restricting it and imposing
> consequential and potential incompatibility on *every* PAP forever after.

It is well known, that there is only one chance to ever define a meaning
for argument specifiers of length > 2.  The TIP does a good job of leaving
"hooks" for further enhancements as options to be added lateron.

The tip does a good job of limiting completely unnecessary and nowhere justified modification to the existing protocol.  The implementation may do a reasonable job of limiting the imposition of overhead incurred by the unnecessary modification so it 'only' impacts every script that has ever used $args (as I understand it, this is not stated in the TIP.)

I don't believe that 'the damage could have been *much* worse, you should have seen the original proposal' is a compelling argument to adopt completely avoidable and nowhere justified arbitrary interference with an existing fundamental Tcl command.

Maybe I just have unrealistic standards. 

It may preclude other means to enhance arg-specs, but given that:
  - "1 or 2 elements" is already taken,
  - decorated param-name is not precluded by the tip, and
  - adding even just a third element would imply presence of a default,
then it seems that any reasonable alternative wouldn't be all that
fundamentally different from 457.

I'd be curious about even rough sketches of alternatives to enhance the
PAP compatibly, that would be *precluded* by acceptance of tip 457.
This is not about laying the burden on proof on you, but it is about
pointing out mistakes in my above reasoning about possible alternatives.

I really don't care what enhanced mechanisms are provided to transform scripts into commands.  I only care that they have a distinct and new name, because ::proc is already taken, and because the overloading of the name ::proc serves no purpose that anyone has articulated anywhere.

> I think Alexandre's counter-proposal for a generic arg-handler at runtime
> is better for precisely that reason.  Any user of the other CGCs  could
> elect to follow FA's arg protocol [FAP] if the author wished.

Having non-introspectable proc foo args ... is just not the objective.

Oh, you have determined that [eatargs] is absolutely not susceptible to introspection?  Remarkable.  How did you determine that?

No, FA seems not to mention it - anymore? - ... no idea where I picked that
up ...  anyway, as I said it wasn't for them, anyway, but because introspect-
ability *should* have been a requirement, imho.

Yeah, introspection is good, motherhood is good, therefore ::proc semantics needs to be changed without any justification.  Is that it?

> > The very point of a TIP is, that the respective author is *not* fine
> > with the workarounds, and *therefore* suggests a change to *improve*
> > Tcl towards *not* needing the workarounds.
> And of course the problem with any improvement is that if it forecloses
> other changes it can be a local maximum.

That's some curse of the world as it is.  If you never change anything
out of fear that it could preclude something not-even-yet-fathomed later,
then it grinds to a halt ways below even the local maximum.

Measure twice, cut once.

> Is this the best of all possible ::procs?  I doubt it, but I don't know,
> and I see no evidence in the TIP that the authors have considered that
> weighty question.

The discussion has passed various stages and suggestions have been
added, refined, dropped. Almost all of my own suggestions have been
replaced by better (as I now understand) alternatives - something that
at least makes me confident that I'm not merely "defending my own stuff".

For what it's worth, I haven't so much as scanned your months of continuous improvement, and I found a pretty amazing hole straight away ... no attempt to justify coopting ::proc.  Imagine what a long-term analysis might find.

> That a given author has elected to implement their proposed command
> improvement in a certain way is really no guarantee that the improvement
> couldn't be improved.

It has been improved a lot during the discussions.
IMHO, another month or so of discussion might still improve it, in
particular for making sure that the other TIP's [info linkedname] actually
can fill the gap of dropped "-varname" (I really hope it can!)

> I see absolutely no argument even attempted that the arbitrary choice to
> modify ::proc in (a way which practically cannot be reversed) is necessary,
> or desirable, or minimal or an improvement to Tcl ...

I'm sorry to read about that physical condition of yours.

So I'm guessing that response means there isn't one.

> > Without "improvements" we still wouldn't have $var, {*}-expansion,
> > and maybe "proc" itself would be an extension to download and compile
> > separately...
> Those were genuine improvements, and *much* more consideration was given to
> their Tcl-wide effects than has been given to this.

"*much* more consideration" was indeed necessary because (at least) $var
was obviously not "compatible" in the strict sense.

> Moreover: {*} went through rapid revision from {expand}, demonstrating that
> its authors were flexible enough to reconsider their initial design choices
> when presented with better ones (see Alexandre's counter-proposal.)

{*} was proposed nearly same time as {expand}.  It was during discussion that
the {expand} was preferred for verbal clarity, but pragmatic rationality
fortunately came back just in time, even if only after the {expand}-vote.

Alexander's counter-proposal isn't "better", because it lacks introspecta-
bility. And i15y is obviously *not* for any FAB, but because it makes sense.

Alexandre's counter-proposal doesn't even address the issue, but addresses the issue of better ways to satisfy the actual stated requirements (to the extent that such a thing exists.)

> If I were you, I wouldn't cite $var in support of your radical language
> mod, because it too easily leads to a discussion about $() and WTF is
> between those parentheses.

Despite the flaws of current $... spec-/implementation, I'm ways happy
to have it, and I've written a TIP to try to amend some of the flaws.
Offered an alternate timeline where $var wasn't added, but instead still
waiting for the perfect change, I'd choose ours.

Me too.  $ wasn't an already-defined sememe which someone felt a desperate need to change coincidentally upon hearing about an ill-defined bounty, though.  So your point isn't apposite.

And I'd prefer a future timeline of enhanced PAP to one of leaving PAP
as it is (in expectation of some utopic perfect enhancement).

Justify why FAP needs to displace PAP in ::proc, or I respectfully disagree.

Colin (wow that was boring)

[Alexandre Ferrieux]

On Sun, May 21, 2017 at 12:58 PM, Andreas Leitgeb <a...@logic.at> wrote:
> Alexander's counter-proposal isn't "better", because it lacks introspecta-
> bility.

Why so ?

proc f {x y z} {
eatargs -allbyname -whatever-other-argspec-gets-TIPped
...
}
f -z 3 -x 0 -y 2

-Alex

[Colin McCormack]

Oh, sorry, I misunderstood the import of your question.

Is it, or is it not the case that your implement adds extra code on the path to plain-old ::proc-style positional binding of actual to formal parameters, or to the args parameter?

I understood you to have stated that it does.  Am I mistaken?  I am unaware that you have tests for the $args case, at least.

Colin.

[Alexandre Ferrieux]

On Sun, May 21, 2017 at 2:56 PM, Alexandre Ferrieux
<alexandre...@gmail.com> wrote:
> On Sun, May 21, 2017 at 12:58 PM, Andreas Leitgeb <a...@logic.at> wrote:
>> Alexander's counter-proposal isn't "better", because it lacks introspecta-
>> bility.
>
> Why so ?
>
> proc f {x y z} {
> eatargs -allbyname -whatever-other-argspec-gets-TIPped
> ...
> }
> f -z 3 -x 0 -y 2

Oops, I meant:

proc f args {
eatargs {your proposal's argspec}

...
}
f -z 3 -x 0 -y 2

That's still introspectable :-)

[Alexandre Ferrieux]

Hi Mathieu,

On Sun, May 21, 2017 at 1:27 PM, Mathieu Lafon <mla...@gmail.com> wrote:
>
> % proc log {
> {level -name level -switch {{debug 0} {error 3}} -default 1}
> {time -name {time timestamp}}
> {message} } {
> if {![info exists time]} {set time [clock seconds]}
> array set levels {0 DEBUG 1 INFO 2 WARN 3 ERROR}
> puts "[clock format $time] : $levels($level) : $message"
> }
>

> % log -level 2 "Warning..."
> Sun Apr 23 20:29:41 UTC 2017 : WARN : Warning...
> % log -timestamp 946684800 -debug -- -Y2K-
> Sat Jan 01 00:00:00 UTC 2000 : DEBUG : -Y2K-
>
> What will it look like with [eatargs]?
> - How are local variables defined? Will they be created by [eatargs] at runtime?
> - How are defined the named arguments? Is it a parameter of [eatargs]
> or are they defined elsewhere?

Here's how I would [eatargs] it :-)

proc log args {
eatargs {{level -name level -switch {{debug 0} {error 3}} -default 1}

{time -name {time timestamp}}
{message} }

...
}

The variables can be thought of as populated by [eatargs] at runtime,
though the CompileProc of [eatargs] will prepare their slots in the
LVT.

-Alex

[Colin McCormack]

On Sun, 21 May 2017, 19:52 Donald Arseneau, <as...@triumf.ca> wrote:

OK, I'm not going to quibble about no impact versus no overhead
versus immeasurable effect.

"Immeasurable" is not a thing, in any actual world.

One thing that did strike me, though, in the recent exchange was
that the new proposed proc would be harder to emulate.  It is
ironic that a feature designed to avoid "scaffolding" prevents
other scaffolding. I don't think that is a show-stopper though.

Yeah, I think it's a significant and unsurmountable drawback to coopting ::proc.

PS to Colin: "invoke a proc" is perfectly clear. Hint: look at
[info proc]. That comment gets you on the threshold of my *plonk*
bin.

Invoke a proc is so clear that it appears to have suffused the thinking and clouded the implementers so they believed that coopting ::proc was a specified requirement.  That was rather my point, as I was searching for some (any) actual justification for the decision to coopt.

The proof of existence of some meaning for a thing doesn't imply that all usages derive meaning from that definition - info proc demonstrates nothing.

May I attain such pellucidity as you attribute to the bounty when I tell you: you know clearly what you can do with your plonk bin.

[Andreas Leitgeb]

On Sun, May 21, 2017 at 03:02:32PM +0200, Alexandre Ferrieux wrote:
> On Sun, May 21, 2017 at 2:56 PM, Alexandre Ferrieux
> <alexandre...@gmail.com> wrote:
> > On Sun, May 21, 2017 at 12:58 PM, Andreas Leitgeb <a...@logic.at> wrote:
> >> Alexander's counter-proposal isn't "better", because it lacks introspecta-
> >> bility.
> > Why so ?

> proc f args {
> eatargs {your proposal's argspec}
> ...
> }
> f -z 3 -x 0 -y 2
> That's still introspectable :-)

Introspectable by which means? Yes, I can call info args in it
and get "args" back, but I would be somewhat surprised if you
seriously called that "introspectable" in this context.

One might also use [info body f] and scan for some "eatargs" in the
first few lines, possibly allowing for comments or injected code
to precede it... well, not my meaning of introspectable, anyway.

[Alexandre Ferrieux]

On Sun, May 21, 2017 at 4:11 PM, Andreas Leitgeb <a...@logic.at> wrote:
> On Sun, May 21, 2017 at 03:02:32PM +0200, Alexandre Ferrieux wrote:
>> On Sun, May 21, 2017 at 2:56 PM, Alexandre Ferrieux
>> <alexandre...@gmail.com> wrote:
>> > On Sun, May 21, 2017 at 12:58 PM, Andreas Leitgeb <a...@logic.at> wrote:
>> >> Alexander's counter-proposal isn't "better", because it lacks introspecta-
>> >> bility.
>> > Why so ?
>> proc f args {
>> eatargs {your proposal's argspec}
>> ...
>> }
>> f -z 3 -x 0 -y 2
>> That's still introspectable :-)
>
> Introspectable by which means? Yes, I can call info args in it
> and get "args" back, but I would be somewhat surprised if you
> seriously called that "introspectable" in this context.
>
> One might also use [info body f] and scan for some "eatargs" in the
> first few lines, possibly allowing for comments or injected code
> to precede it... well, not my meaning of introspectable, anyway.

Yep, the latter's the idea. It is not perfect, but the importance of
i15y wrt the main goal of the TIP is rather secondary too (I doubt
anybody would fight for the remote possibility of a debugger that
would statically validate call sites against named-argspecs).

-Alex

[Andreas Leitgeb]

On Sun, May 21, 2017 at 12:52:56PM +0000, Colin McCormack wrote:
> What I have said is that I see no justification in the tip, nor anywhere
> else, for the decision to make tip457 extend/modify/mutilate ::proc, rather
> than simlly make a parallel command.

Since you've "not seen" it the other umpty hundred times: tip457 targets
Tcl_CreateProc exactly to extend/modify/mutilate *not only* ::proc, but
also ::apply, oo-methods/constructors, and extensions using Tcl_Create-
Proc to implement their own variants of custom CGC leaving the argument
initialization bit to Tcl. I've yet to see a proc-alike in the wild that
isn't directly or indirectly built on Tcl_CreateProc().

You may (or more likely not) agree to this, but you cannot truthfully
say, that you "saw" no such justification "anywhere else", unless, you have
that physical condition of being blind and have a screen-reader to read
these messages. In that case, it would be fair to clarify if you not
only saw no justification but actually don't see anything at all.

> The important relationship I am referring to is that, as the PAP *calling*
> protocol comprises a list of values, any imposition of a protocol involving
> named parameters, call by name, etc, results in the possibility of creation
> of commands by ::proc which simply cannot be simulated (or cannot simply)
> be stimulated with other CGCs such as ::apply, ::coroutine, etc.

You lost me shortly after "imposition". In case you attempt to put it in
easier words, maybe you could also provide an example for such a case of
proc-s(t?)imulation in the wild. If it looks like a reasonable usecase,
then I don't see how this wouldn't be convincing. But since you mention
s(t?)imulating it with apply, then you seem to once again have forgotten
that apply is also covered by the tip, and that coroutines "inherit"
the argument parsing from the coroutinized command/proc/lambda, and that
the result of yieldto is already non-trivial (and also not meant) to be
used for initializing local variables, as the local stackframe is no
longer fresh at that time.

> > If we were to enhance list syntax (how strings get parsed to lists), we'd
> > rather not add a new xlindex, xlappend, xlset, xforeach,... but instead
> > enhance the appropriate Tcl_...List... function.
> I don't think I would recommend 'enhancing' list parsing by modifying the
> [list] command,

As if [list] even *did* any list *parsing*...

> and I would strenuously counsel against doing so if your
> objective was to create dicts.

I expected that you'd try to twist the point towards absurdity, and
that's why I put an explanation to what would be enhanced: "how strings
get parsed to lists". That obviously doesn't involve dicts, except as
a red herring.

> Perhaps my usage of "protocol" in this has been unclear. I am using it in
> the same sense Occam used it.

You're certainly aware that William of Ockham wouldn't have accepted the
excessively redundant form of $var, when there was already [set var].
A "protocol" that is perfect by Occam's Razor is not necessarily the one
that anyone would enjoy directly using.

> a reasonable job of limiting the imposition of overhead incurred by the
> unnecessary modification so it 'only' impacts every script that has ever
> used $args (as I understand it, this is not stated in the TIP.)

Ah, nice! Yet another wild speculation far off the facts...

> > Having non-introspectable proc foo args ... is just not the objective.
> Oh, you have determined that [eatargs] is absolutely not susceptible to
> introspection? Remarkable. How did you determine that?

Because proc-bodies are (except for machine- and "int"- limits) algorithms
specified in a turing-complete language, so, unless the param-spec is
reflected rather than merely duplicated somewhere else, it can not be
considered generally available.

> > No, FA seems not to mention it - anymore? - ... no idea where I picked that
> > up ...

I think I remember now, that Karl Lehenbauer confirmed the requirement
in a message on this list, but apparently didn't (yet?) put it on the
bounties-page.

> Measure twice, cut once.

Yes, during the time it's being discussed it has been measured 100
times, and may yet need another 100 measurements. It won't get all
the [expr {Inf}] measurements, though, that you might consider
necessary for an exactly 100% optimum cut.

> For what it's worth, I haven't so much as scanned your months of continuous
> improvement, and I found a pretty amazing hole straight away ... no attempt
> to justify coopting ::proc. Imagine what a long-term analysis might find.

Would be interesting if you actually *had* found any "amazing hole"
and were not just putting your disapproval of the tip's scope in fancy
words.

> Alexandre's counter-proposal doesn't even address the issue,

Yes, that's exactly my perceived problem with his [eatargs].

> > And I'd prefer a future timeline of enhanced PAP to one of leaving PAP
> > as it is (in expectation of some utopic perfect enhancement).
> Justify why FAP needs to displace PAP in ::proc, or I respectfully disagree.

That has also been said a lots of times:

The point is to improve the natural use of "proc" and the other CGCs.
The point is to make named options a "normal" part of Tcl. As "normal"
as defining and using procedures in the first place. That's why it is
indeed an TIP, and not a CAP (Command Addition Proposal).

> Colin (wow that was boring)

Would be less boring if you didn't just play the "I see no justification ..."
clip in auto-repeat mode.

[Colin McCormack]

Wow.  Obtuse.

On Mon, 22 May 2017 at 01:13 Andreas Leitgeb <a...@logic.at> wrote:

On Sun, May 21, 2017 at 12:52:56PM +0000, Colin McCormack wrote:
> What I have said is that I see no justification in the tip, nor anywhere
> else, for the decision to make tip457 extend/modify/mutilate ::proc, rather
> than simlly make a parallel command.

Since you've "not seen" it the other umpty hundred times: tip457 targets
Tcl_CreateProc exactly to extend/modify/mutilate *not only* ::proc, but
also ::apply, oo-methods/constructors, and extensions using Tcl_Create-
Proc to implement their own variants of custom CGC leaving the argument
initialization bit to Tcl.  I've yet to see a proc-alike in the wild that
isn't directly or indirectly built on Tcl_CreateProc().

I'm not sure you understand what I mean by 'justification.'

In general, merely stating that you've done something one way does not serve as a justification for having done something that way.

It's a Tcl Improvement Process, not a 'pick a random command and add some stuff to it' process.

 

You may (or more likely not) agree to this, but you cannot truthfully
say, that you "saw" no such justification "anywhere else", unless, you have
that physical condition of being blind and have a screen-reader to read
these messages. In that case, it would be fair to clarify if you not
only saw no justification but actually don't see anything at all.

Why is this so hard?  You haven't justified the provision of named arguments by modifying ::proc.  Nowhere.  Nowhere at all, that I can see.

 

> The important relationship I am referring to is that, as the PAP *calling*
> protocol comprises a list of values, any imposition of a protocol involving
> named parameters, call by name, etc, results in the possibility of creation
> of commands by ::proc which simply cannot be simulated (or cannot simply)
> be stimulated with other CGCs such as ::apply, ::coroutine, etc.

You lost me shortly after "imposition". In case you attempt to put it in
easier words, maybe you could also provide an example for such a case of
proc-s(t?)imulation in the wild. If it looks like a reasonable usecase,
then I don't see how this wouldn't be convincing. But since you mention
s(t?)imulating it with apply, then you seem to once again have forgotten
that apply is also covered by the tip, and that coroutines "inherit"
the argument parsing from the coroutinized command/proc/lambda, and that
the result of yieldto is already non-trivial (and also not meant) to be
used for initializing local variables, as the local stackframe is no
longer fresh at that time.

Every time I use a coroutine, I am simulating a proc.

> > If we were to enhance list syntax (how strings get parsed to lists), we'd
> > rather not add a new xlindex, xlappend, xlset, xforeach,... but instead
> > enhance the appropriate Tcl_...List... function.
> I don't think I would recommend 'enhancing' list parsing by modifying the
> [list] command,

As if [list] even *did* any list *parsing*...

You're right.  [list] merely relies upon the native PAP calling protocol to provide it a list.

 

Note, btw, I *also* wouldn't modify [split] to 'enhance' list ... not that your proposal actually enhances scripts - as I understand it scripts aren't modified at all.

 

> and I would strenuously counsel against doing so if your
> objective was to create dicts.

I expected that you'd try to twist the point towards absurdity, and
that's why I put an explanation to what would be enhanced: "how strings
get parsed to lists".  That obviously doesn't involve dicts, except as
a red herring.

I prefer to see myself as making absurdity patent.

One might consider dicts to be a kind of enhanced list ... certainly any dict has a representation as a list.  In fact, there were once quite a few commands (maybe still are) for a type called keylists (in TclX)   They functioned a lot like dicts, and they (listen to this bit) had their own constructors ... they didn't use [list] or [split], they also looked much as your counter-example does ... so much so that I thought you were being arch.

 

> Perhaps my usage of "protocol" in this has been unclear.  I am using it in
> the same sense Occam used it.

You're certainly aware that William of Ockham wouldn't have accepted the
excessively redundant form of $var, when there was already [set var].
A "protocol" that is perfect by Occam's Razor is not necessarily the one
that anyone would enjoy directly using.

The language, mate, not the historical personage.  FFS.

 

> a reasonable job of limiting the imposition of overhead incurred by the
> unnecessary modification so it 'only' impacts every script that has ever
> used $args (as I understand it, this is not stated in the TIP.)

Ah, nice! Yet another wild speculation far off the facts...

I believe I derived this from something your co-author said.  I may of course be mistaken.

You're able to assure me, from your awareness of the facts, that there is no execution overhead for scripts which do not use the FAP?

 

> > Having non-introspectable proc foo args ... is just not the objective.
> Oh, you have determined that [eatargs] is absolutely not susceptible to
> introspection?  Remarkable.  How did you determine that?

Because proc-bodies are (except for machine- and "int"- limits) algorithms
specified in a turing-complete language, so, unless the param-spec is
reflected rather than merely duplicated somewhere else, it can not be
considered generally available.

Alexandre disagrees, elsewhere.

> > No, FA seems not to mention it - anymore? - ... no idea where I picked that
> > up ...
I think I remember now, that Karl Lehenbauer confirmed the requirement
in a message on this list, but apparently didn't (yet?) put it on the
bounties-page.

Wow.  So *this* is the justification for co-opting ::proc ???  Just Wow.

 

> Measure twice, cut once.

Yes, during the time it's being discussed it has been measured 100
times, and may yet need another 100 measurements.  It won't get all
the [expr {Inf}] measurements, though, that you might consider
necessary for an exactly 100% optimum cut.

I think you've missed some stuff ... like 'why'.

 

> For what it's worth, I haven't so much as scanned your months of continuous
> improvement, and I found a pretty amazing hole straight away ... no attempt
> to justify coopting ::proc.  Imagine what a long-term analysis might find.

Would be interesting if you actually *had* found any "amazing hole"
and were not just putting your disapproval of the tip's scope in fancy
words.

No justification that modding ::proc improves Tcl ... not a hole?

No comparison with perfectly servicable alternatives ... not a hole?

 

> Alexandre's counter-proposal doesn't even address the issue,

Yes, that's exactly my perceived problem with his [eatargs].

You're bombastically asserting it's not possible, sans evidence.  I'm starting to see a pattern here.

 

> > And I'd prefer a future timeline of enhanced PAP to one of leaving PAP
> > as it is (in expectation of some utopic perfect enhancement).
> Justify why FAP needs to displace PAP in ::proc, or I respectfully disagree.

That has also been said a lots of times:

The point is to improve the natural use of "proc" and the other CGCs.
The point is to make named options a "normal" part of Tcl. As "normal"
as defining and using procedures in the first place. That's why it is
indeed an TIP, and not a CAP (Command Addition Proposal).

So the scope has expanded markedly without any justification of the modifications proposed being advantageous to Tcl.  Got it.

> Colin (wow that was boring)

Would be less boring if you didn't just play the "I see no justification ..."
clip in auto-repeat mode.

Might be less boring if you evinced any evidence of having thought in terms of having to justify a fundamental change, beyond "I think Karl said in a ML somewhere that he would like."

Might be interesting to read an actual justification, beyond "but that's what we did."  OTOH, maybe not, too ... I'm speculating.

Colin

 

[Mathieu Lafon]

On Sun, May 21, 2017 at 3:01 PM, Colin McCormack <mcc...@gmail.com> wrote:
> Is it, or is it not the case that your implement adds extra code on the path
> to plain-old ::proc-style positional binding of actual to formal parameters,
> or to the args parameter?
>
> I understood you to have stated that it does. Am I mistaken? I am unaware
> that you have tests for the $args case, at least.

I don't know why you think that. There is nothing special with args in
the handling of old-style proc.

[Colin McCormack]

I beg your pardon.  I must have been mistaken.  There's also no execution-time overhead in your implementation for commands which do not use your proposed facilities imposed by your support of those facilities?

[Mathieu Lafon]

On Sun, May 21, 2017 at 12:21 PM, Donald Arseneau <as...@triumf.ca> wrote:

> In the case of 1, I don't think there is a need to parse the
> argument list for simple debugging goals, and if argument tracking
> is needed then it is feasible to do so (with work).

Agreed.

> For 2, the need
> to emulate maned parameters could be a lot of work, and a barrier
> too difficult to overcome; but such revised commands don't need to
> use the name "proc" and then wouldn't need to support named arguments.
> That seems better than restricting the scope of the core "proc".

I dont think you need to emulate named parametrer. If you want to wrap
a proc with named parameter, you would probably use the same arg
specifiers in the new proc.

Do you have an example to share, where you think this can be a problem?

-- Mathieu

[Andreas Leitgeb]

On Sun, May 21, 2017 at 04:12:29PM +0000, Colin McCormack wrote:
> Every time I use a coroutine, I am simulating a proc.

You probably mean "resume a coroutine" here, and my empathy is quite
limited for this abuse of coroutines.

> > > Perhaps my usage of "protocol" in this has been unclear. I am using it
> > > in the same sense Occam used it.

> > You're certainly aware that William of Ockham ...

> The language, mate, not the historical personage. FFS.

99.9% of mentions of Occam are about the historical person and his "razor".
Also, the language occam seems to be written with lowercase "o".

If you mean anything else than the algorithm by which call-arguments
get assigned to formal parameter initializations in the proc's stack
frame, you might want to use English or Tcl language to describe it.

This protocol is the TIP's target. It is so, because it is an
enhancement in the real sense (unlike your ridiculous claim of
dicts being enhancements of lists) and it is meant (by the TIP-
authors and supporters) to become the standard PAP to make it
easy even for new users to create more conveniently callable
procedures/lambdas/oo-methods/... and not something special that
you need a special command for and that cannot be used in the
other CGC contexts or requires an extra boilerplate in the body.

> > > a reasonable job of limiting the imposition of overhead incurred by the
> > > unnecessary modification so it 'only' impacts every script that has ever
> > > used $args (as I understand it, this is not stated in the TIP.)
> > Ah, nice! Yet another wild speculation far off the facts...
> I believe I derived this from something your co-author said. I may of
> course be mistaken.
> You're able to assure me, from your awareness of the facts, that there is
> no execution overhead for scripts which do not use the FAP?

Tcl_CreateProc currently creates some internal object to represent a
defined procedure/lambda/oo-method/... that upon invocation (calling
one of the structure's function pointers) will create a new stackframe,
do the "PAP" and run the body.

With TIP457, Tcl_CreateProc will classify the param-spec as either being
compatible with old style (then create the standard object for it), or
as actively using the new features, in which case it creates a different
object with a pointer to a different function that implements the enhanced
PAP.

There might be fractions of a millisecond spent on determining compati-
bility of the param-spec during execution of the [proc] command itself,
but once the internal object is there, it doesn't even know that there
is also another one.

So, since the code path doesn't change for unchanged procedures, what
slowdown would you expect?

> > > > Having non-introspectable proc foo args ... is just not the objective.
> > > Oh, you have determined that [eatargs] is absolutely not susceptible to
> > > introspection? Remarkable. How did you determine that?
> > Because proc-bodies are (except for machine- and "int"- limits) algorithms
> > specified in a turing-complete language, so, unless the param-spec is
> > reflected rather than merely duplicated somewhere else, it can not be
> > considered generally available.
> Alexandre disagrees, elsewhere.

Nope, Alexandre merely considers introspection as not important and some
hacks to examine the body in sunny-weather cases as sufficient.

> > > > No, FA seems not to mention it - anymore? - ... no idea where I picked
> > > > that up ...
> > I think I remember now, that Karl Lehenbauer confirmed the requirement

> > in a message on this list, ...

> Wow. So *this* is the justification for co-opting ::proc ??? Just Wow.

Nope, it was just about why I even assumed FA's interest in introspection.

> > > Measure twice, cut once.
> > Yes, during the time it's being discussed it has been measured 100
> > times, and may yet need another 100 measurements. It won't get all
> > the [expr {Inf}] measurements, though, that you might consider
> > necessary for an exactly 100% optimum cut.
> I think you've missed some stuff ... like 'why'.

Because of my subjective judgement of the case. That's why. And your
judgement is different, as is hard not to notice. And that's where
our judgements are both irrelevant, anyway, because we're both not in
the TCT.

> No justification that modding ::proc improves Tcl ... not a hole?
> No comparison with perfectly servicable alternatives ... not a hole?

Nope. Both are subjective judgements of the options: to be able to
make option/switch-accepting procedures easily is either favorable,
or it is not.
... and about importance of sound i15y.

In the meantime some details may be changed about the TIP in hope
to nudge the scale towards favorability by the TCT majority.

> Might be interesting to read an actual justification, beyond "but that's
> what we did." OTOH, maybe not, too ... I'm speculating.

I'm still struggling about what defines a "justification" for you.
Maybe you could add a sample justification that you would use for
adding $var if it weren't already there. I might then find something
on par for why named arguments should be done in Tcl_CreateProc()
and not in some eatargs or faproc.

[Andreas Leitgeb]

On Sun, May 21, 2017 at 04:18:13PM +0200, Alexandre Ferrieux wrote:
> On Sun, May 21, 2017 at 4:11 PM, Andreas Leitgeb <a...@logic.at> wrote:
> > On Sun, May 21, 2017 at 03:02:32PM +0200, Alexandre Ferrieux wrote:
> >> On Sun, May 21, 2017 at 2:56 PM, Alexandre Ferrieux
> >> <alexandre...@gmail.com> wrote:
> >> > On Sun, May 21, 2017 at 12:58 PM, Andreas Leitgeb <a...@logic.at> wrote:
> >> >> Alexander's counter-proposal isn't "better", because it lacks introspecta-
> >> >> bility.

> > [... info body parsing ...]

> Yep, the latter's the idea. It is not perfect,

This "not perfect" seems like a euphemism par excellence to me.

> but the importance of
> i15y wrt the main goal of the TIP is rather secondary too (I doubt
> anybody would fight for the remote possibility of a debugger that
> would statically validate call sites against named-argspecs).

Have you read/heard about tools like e.g. "nagelfar"?

[Colin McCormack]

Once more into the breach...

On Mon, 22 May 2017, 03:28 Andreas Leitgeb, <a...@logic.at> wrote:

On Sun, May 21, 2017 at 04:12:29PM +0000, Colin McCormack wrote:
> Every time I use a coroutine, I am simulating a proc.

You probably mean "resume a coroutine" here, and my empathy is quite
limited for this abuse of coroutines.

I meant what I said: use a coroutine.  Use of coroutines often entails their resumption.

Empathy ... it is fortunate indeed that Tcl's most effective means of interacting with asynchronous events does not depend on yours, or upon you understanding of its efficacy.

> > > Perhaps my usage of "protocol" in this has been unclear.  I am using it
> > > in the same sense Occam used it.
> > You're certainly aware that William of Ockham ...
> The language, mate, not the historical personage.  FFS.

99.9% of mentions of Occam are about the historical person and his "razor".
Also, the language occam seems to be written with lowercase "o".

ORLY. Back in the day, I think we used uppercase.

If you mean anything else than the algorithm by which call-arguments
get assigned to formal parameter initializations in the proc's stack
frame, you might want to use English or Tcl language to describe it.

The usage of 'protocol' is specialised, Occam makes much of the syntax by which 'arguments' are 'passed.'  Since I've adopted their term, I thought it worth acquainting you with it to avoid confusion.  That doesn't seem possible.

Another term of art you might find useful is 'binding' for the process of, as you say, associating actual with formal parameters, forming a binding or a scope.

Doesn't seem that you looked very far afield when seeking prior art.  Shame, that.

This protocol is the TIP's target.

Only for proc and proc-like commands, apparently.  That limitation would be a great loss of facility to Tcl.

It is so, because it is an
enhancement in the real sense (unlike your ridiculous claim of
dicts being enhancements of lists)

They're not?  Well, I confess that I don't know what you were referring to, then, when *you* suggested a possible enhancement to list.  What did you mean to indicate, then?

I tire of trying to help you make sense of what you mean, only to be snarked at in response.

and it is meant (by the TIP-
authors and supporters) to become the standard PAP

The standard PAP, and different from and incompatible with all other Command Argument Protocols [CAP].

You appear to be unaware of other CAPs ... perhaps because they're not attractive to beginners, but I would like to draw your attention to the protocol by which one can substitute a C function for a Tcl proc (often for performance reasons.)

It would be necessary, I think, to describe clearly which of the FAProc forms can be expected to convert directly to C functions,  as with SWIG for example or by hand, and which would be newly incapable of such transformation.  Perhaps you could add this onerous parser to your [info] introspection ... maybe then SWIG etc could at least error on encountering newly-incompatible forms.

to make it easy even for new users

Only new users, I think, and only when they know they'll never need to convert them to C ... but who am I kidding ... they're never going to do that.

to create more conveniently callable procedures/lambdas/oo-methods/...

How is it 'more convenient' exactly?  I have written thousands, tens of thousands, of procs, lambdas, methods and have never once wanted for convenience.

See, this would be something akin to a justification for something - actually substantiating your wild claims.

and not something special that you need a special command for and that cannot be used in the other CGC contexts or requires an extra boilerplate in the body.

Your proposed ::FAProc *is* a special command.  Your proposal is *not* available to other CGC contexts (such as coroutine resumption) nor to commands written in or translated into C.

You could implement your ::FAProc in C over ::eatargs, and it would function just fine ... but no, you have to *demand* that you can modify ::proc ... because the guy you expect to pay you money "mentioned it somewhere in a mailing list post" and because "won't SOMEBODY think of the children?" (AKA hypothetical new users who wouldn't be able to comprehend variant PAP *unless* it were hard-coded into ::proc ... the idea of them learning it in a command with a different name is unthinkable!)

This is risible.  There is no justification for new users ... they're not going to care what the command is called when they learn a new PAP, the *only* justification for your desire to modify ::proc is your master's voice, and perhaps what seems a megalomaniacal refusal to understand that others' use of Tcl might differ from what you can imagine.

> > > a reasonable job of limiting the imposition of overhead incurred by the
> > > unnecessary modification so it 'only' impacts every script that has ever
> > > used $args (as I understand it, this is not stated in the TIP.)
> > Ah, nice! Yet another wild speculation far off the facts...
> I believe I derived this from something your co-author said.  I may of
> course be mistaken.
> You're able to assure me, from your awareness of the facts, that there is
> no execution overhead for scripts which do not use the FAP?

Tcl_CreateProc currently creates some internal object to represent a
defined procedure/lambda/oo-method/... that upon invocation (calling
one of the structure's function pointers) will create a new stackframe,
do the "PAP" and run the body.

With TIP457, Tcl_CreateProc will classify the param-spec as either being
compatible with old style (then create the standard object for it), or
as actively using the new features, in which case it creates a different
object with a pointer to a different function that implements the enhanced
PAP.

There might be fractions of a millisecond spent on determining compati-
bility of the param-spec during execution of the [proc] command itself,
but once the internal object is there, it doesn't even know that there
is also another one.

So, since the code path doesn't change for unchanged procedures, what
slowdown would you expect?

Yep, no, it seems I misunderstood what your coauthor said and there's no execution-time imposition on standard PAP.

> > > > Having non-introspectable proc foo args ... is just not the objective.
> > > Oh, you have determined that [eatargs] is absolutely not susceptible to
> > > introspection?  Remarkable.  How did you determine that?
> > Because proc-bodies are (except for machine- and "int"- limits) algorithms
> > specified in a turing-complete language, so, unless the param-spec is
> > reflected rather than merely duplicated somewhere else, it can not be
> > considered generally available.
> Alexandre disagrees, elsewhere.

Nope, Alexandre merely considers introspection as not important and some
hacks to examine the body in sunny-weather cases as sufficient.

Of course [eatargs] can support whatever introspection your proposal does ... isn't that immediately obvious to you?

It's a total distraction from what is important.

> > > > No, FA seems not to mention it - anymore? - ... no idea where I picked
> > > > that up ...
> > I think I remember now, that Karl Lehenbauer confirmed the requirement
> > in a message on this list, ...
> Wow.  So *this* is the justification for co-opting ::proc ???  Just Wow.
Nope, it was just about why I even assumed FA's interest in introspection.

So is there any documented requirement that ::FAProc be shoe-horned into ::proc, and ::proc go away, or did it just seem like a good idea to you at the time?

> > > Measure twice, cut once.
> > Yes, during the time it's being discussed it has been measured 100
> > times, and may yet need another 100 measurements.  It won't get all
> > the [expr {Inf}] measurements, though, that you might consider
> > necessary for an exactly 100% optimum cut.
> I think you've missed some stuff ... like 'why'.

Because of my subjective judgement of the case. That's why.  And your
judgement is different, as is hard not to notice. And that's where
our judgements are both irrelevant, anyway, because we're both not in
the TCT.

Your judgement may be irrelevant (some other adjectives also spring to mind,) but my judgement tells me that what is lost by your pig-headed insistence upon placing your stamp on ::proc rather than making a new command and/or using ::eatargs is quite bad for Tcl.

My judgement also tells me that before adopting radical language modification is must be explicitly justified in terms of cost/benefit.

My judgement also tells me that these issues should be thoroughly ventilated before even floating such a proposal, but this was not done in this case.

> No justification that modding ::proc improves Tcl ... not a hole?
> No comparison with perfectly servicable alternatives ... not a hole?

Nope. Both are subjective judgements of the options: to be able to
make option/switch-accepting procedures easily is either favorable,
or it is not.
... and about importance of sound i15y.

I have no idea what i15y stands for.

In the meantime some details may be changed about the TIP in hope
to nudge the scale towards favorability by the TCT majority.

The one missing detail that makes all the difference is humility ... you ought not to be changing ::proc without a compelling justification, but ought to create a new name for your experimental variant.

> Might be interesting to read an actual justification, beyond "but that's
> what we did."  OTOH, maybe not, too ... I'm speculating.

I'm still struggling about what defines a "justification" for you.
Maybe you could add a sample justification that you would use for
adding $var if it weren't already there.  I might then find something
on par for why named arguments should be done in Tcl_CreateProc()
and not in some eatargs or faproc.

You're struggling with "a dictionary"?  You would like some pointers?  The verb "to justify" is not a neologism ... would it help if I called it j5y?

BTW, I wouldn't have added $var in the way it was added.

Colin

[Colin McCormack]

I'm sorry, I made a mistake.  Of course, SWIG goes from inspection of C to generation of Tcl interface.  It wouldn't be necessary to have introspective detection of C-incompatible forms for SWIG's benefit, but only perhaps for the benefit of facilities which transform Tcl proc into other Tcl command forms.  I think the point stands - proposed introspection is lacking in essentials, largely because the only command form considered was ::proc, and the importance of congruence between command forms is not obvious to a person bound to consider the point of view only of a hypothetical beginner (and, particularly, a magic hypothetical beginner who could *only* learn a new argument protocol if it were associated with a command called ::proc.)

Colin

[Peter da Silva]

I guess you haven’t been following the discussion. For procs not using the new syntax there is no increase in overhead. The new syntax has been carefully designed to not conflict with existing procs in any way. Unless an argument has three or more elements the new code will never be called at any point.

 

From: Colin McCormack <mcc...@gmail.com>
Date: Friday, May 19, 2017 at 10:11 PM
To: Tcl Core List <tcl-...@lists.sourceforge.net>
Subject: [TCLCORE] There is no justification for modifying [proc] for named parameters

 

I don't think modifying [proc] to support a particular protocol for names args is a good idea.

 

I think the protocol [proc] currently has for args is sufficient and I don't think any extension of it is warranted, nor required by the Flight Aware Bounties [FABs].

 

I think that the challenges outlined by FABs would be amply met by defining a different command FAProc in C and packing whatever functionality is required into FAProc.

 

I think that the notion that [proc] needs to be extended (at a runtime cost and at documentation and implementation complexity cost) to cover a particular calling protocol is really a co-opting of the [proc] name, and not an addition of functionality.

 

I think that the remedy for "but I want *all* my proc to be FAProc" is [rename].  I am not aware of what, in the FABs, actually require proc to be pre-[renamed] as it were.

 

I am not aware of what benefit there is in adding processing cost to traditional proc, either for the users of FAProc or the users of proc.

 

I will now closely paraphrase the FABs to try to understand why redefining the verb proc might gain FlightAware (or anyone) anything substantive.

 

I paraphrase the FABs for clarity:

 

"A first class, high-performance, non-hackish way to do named parameters" which "must be first class, i.e. can't be a proc that wraps procs with a bunch of scaffolding" and a Named Parameter Protocol (example FAProc declaration given) which would 'replace' (traditional proc declaration given) and which is to be (and here's the kicker) "implemented as a first class thing in the Tcl code" and not to be implemented as "some proc that generates a proc or something lame like that"

 

The bounty is attracted to a "first class thing" which implements a names parameter protocol, which is not itself implemented as a call to proc 'or something lame like that' ... vague much?

 

Analysis:  any "first class thing" could be either a command or some kind of modification to the parser which applied only to calls to the proc command.  I'm going to assume the former, because the latter seems particularly poorly conceived.

 

It seems to me that the 'replace' part of the bounty specification is amply satisfied by [replace proc AFProc] and that the AFProc command, if implemented in C or some other method which is not "lame" would satisfy the bounty requirements.

 

The requirements of the bounty seem to have been reinterpreted somehow as an imperative to modify the implementation of proc.  They are not.

 

This re-implementation will by its nature drag on proc, will by its nature slow proc definitions which don't use the extended facilities specified by AFProc. This overhead would arise and be imposed for no good reason, but simply because the bounty was poorly phrased.

 

Unless FA get to pay $10k to impose new overhead on every proc call ever, for no concomitant benefit, I think the idea of modifying [proc] to save FA (et al) the time it takes to run [rename] is a *very* bad idea.  I should state, clearly, that I don't think FA get to pay $10k to impose new overhead, nor that FA want that, nor that they would obtain any benefit at all from doing that ... I think they just got carried away with how implementing language extensions in Tcl is 'lame' (which I doubt) and forgot to properly specify what they mean by "first class thing."

 

I have no personal desire to prevent FA from getting their "first class Tcl thing" which isn't "lame like a proc", but I really don't feel any impetus to impose a FA performance load on every [proc] call so their requirements might be satisfied in what seems to be the lamest way possible.

 

Colin

 

[Colin McCormack]

I haven't, and you're right: I was wrong in thinking that there was an execution overhead.

Colin.

[Peter da Silva]

On 5/20/17, 11:16 AM, "Andreas Leitgeb" <a...@logic.at> wrote:
> Without "improvements" we still wouldn't have $var, {*}-expansion, and maybe "proc" itself would be an extension to download and compile separately...

I’d like to add, without improvements suggested and in some cases implemented by Karl and myself, we wouldn’t even have file I/O in Tcl. The version of Tcl described here already had our work in it, in the “stream” command that inspired the file I/O commands.

http://wiki.tcl.tk/6046

[Colin McCormack]

Hi Peter,

I don't know why you feel the need to add that.  That TclX has some cool stuff in it has no bearing on whether *this* proposal has merit, or what merit it has.

Colin.

[Donald G Porter]

On 05/21/2017 06:58 AM, Andreas Leitgeb wrote:
> Obviously, most of them have no relation whatsoever to the "PAP" - the
> "::proc argument protocol", which for now is shared by apply and
> oo-methods, and potentially by any extension that uses Tcl_CreateProc().

A nitpick, but an important one, I believe.

The routine is TclCreateProc(). It is not public. Neither is the
Proc struct at the heart of so much.

We've also never exported CallFrame manipulation routines, so many
extensions building on Proc do so not because they obviously need
to be Proc-like things, but because that's the one semi-blessed
access route to Tcl's CallFrame management facilities.

Tcl's failure to export public facilities allowing extensions to
create their own CallFrame uses or their own [proc] variants is a
contributing factor in this discussion.

Changing the core [proc] has greater impact on more players with
a less partitioned impact than it might have in an alternative history.

OTOH, the fact that the argument processing machinery has always been
kept "private" is what makes consideration of TIP 457 ideas even
possible.

--
| Don Porter Applied and Computational Mathematics Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|