2dsa | !2dsa ?

[Rod Kay]

Hello all,

I've heard that the Distributed Systems Annex (DSA) may be dropped from the Ada standard soon. Can anyone confirm this ?

I've been using the Polyorb implementation of DSA for some time and find it very useful. The way in which it abstracts away socket 'plumbing' details makes for very simple/understandable comms.

[Randy Brukardt]

"Rod Kay" <roda...@gmail.com> wrote in message
news:d3f3b7fb-e051-428b...@googlegroups.com...

> Hello all,
>
> I've heard that the Distributed Systems Annex (DSA) may be dropped from
> the Ada standard soon. Can anyone confirm this ?

Annex E remains in the proposed Ada 202x standard.

Compiler support, of course, is up to vendors. Dunno if anyone is still
supporting it.

> I've been using the Polyorb implementation of DSA for some time and
> find
> it very useful. The way in which it abstracts away socket 'plumbing'
> details makes
> for very simple/understandable comms.

That was the promise, not sure it ever really was realized. Since the Annex
was weakened enough that third-party support isn't really possible anymore
(necessary to allow it to be used with current middleware), it's really a
vendor-specific thing these days.

Randy.

[Dmitry A. Kazakov]

On 2020-12-23 02:32, Randy Brukardt wrote:
> "Rod Kay" <roda...@gmail.com> wrote in message
> news:d3f3b7fb-e051-428b...@googlegroups.com...
>> Hello all,
>>
>> I've heard that the Distributed Systems Annex (DSA) may be dropped from
>> the Ada standard soon. Can anyone confirm this ?
>
> Annex E remains in the proposed Ada 202x standard.
>
> Compiler support, of course, is up to vendors. Dunno if anyone is still
> supporting it.

It should be moved to the user level. As specified in the Annex there
seems no obvious way to provide a user-defined transport for DSA, and
there seems no way to have different implementations of DSA in the same
program.

>> I've been using the Polyorb implementation of DSA for some time and
>> find
>> it very useful. The way in which it abstracts away socket 'plumbing'
>> details makes
>> for very simple/understandable comms.

Yes, but too simple to be universally useful.

> That was the promise, not sure it ever really was realized. Since the Annex
> was weakened enough that third-party support isn't really possible anymore
> (necessary to allow it to be used with current middleware), it's really a
> vendor-specific thing these days.

Yes, I always wished to include DSA support based on various
communication protocols I have implemented in Ada, rather than plain
sockets. E.g. I have a ready-to-go DSA implementation for interprocess
communication over shared memory, but no idea how to make GNAT aware of
it. Or AQMP and ASN.1 look like a straightforward candidate as a DSA
transport as they have detailed type description systems to map Ada objects.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

[Maxim Reznik]

I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)

https://github.com/reznikmm/garlic/tree/web_socket

[Dmitry A. Kazakov]

My question is how to proceed without GLADE/Garlic etc. I have DSA
implemented, e.g. System.RPC. I need GNAT to accept it as such.

In a more distant perspective I need a work-around of stream attributes.
They are non-portable, so there must an option to replace them for DSA
and/or provide a non-stream parameter marshaling when the transport is a
higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For
these you must map Ada types into the types defined by the protocol.

Without this DSA is pretty much pointless.

[Rod Kay]

[Rod Kay]

Apologies for the prior empty post.

Is it likely that the ARG might address the aformentioned issues ?

[Randy Brukardt]

"Rod Kay" <roda...@gmail.com> wrote in message

news:30f2c0d8-58c2-4ca9...@googlegroups.com...

>Apologies for the prior empty post.
>
>Is it likely that the ARG might address the aformentioned issues ?

As of now, it doesn't appear that there would be any point. Annex E is an
optional annex, and so far as we're aware, no compiler vendor has any plans
for increasing support for that annex. So the ARG could change the annex but
it seems unlikely that any changes would make it into implementations.
(We've been told not to expect even the implementation of bugs fixes
included in Ada 202x, even from the vendor that originally requested the bug
fixes.)

Randy.

[Dmitry A. Kazakov]

Why there should be any vendor support in the first place? Why not to
redefine it as a set of abstract tagged types allowing custom user
implementations like storage pools and streams do?

The idea of having an IDL, statically assigned partitions, linking
everything together before start is not the way the distributed systems
are designed and work today. CORBA died for a reason.

[Luke A. Guest]

On 29/12/2020 14:56, Dmitry A. Kazakov wrote:

>> for increasing support for that annex. So the ARG could change the
>> annex but
>> it seems unlikely that any changes would make it into implementations.
>> (We've been told not to expect even the implementation of bugs fixes
>> included in Ada 202x, even from the vendor that originally requested
>> the bug
>> fixes.)
>
> Why there should be any vendor support in the first place? Why not to
> redefine it as a set of abstract tagged types allowing custom user
> implementations like storage pools and streams do?

Would the compiler still need any support for this or would it just be a
set of interfaces at library level?

[Dmitry A. Kazakov]

Yes, because the idea is to have remote objects and remote calls looking
exactly same as local objects and local calls.

So the compiler must translate a call to an RPC to a call to some user
primitive operation like System.RPC does. The operation would have a
controlling parameter "connection" or "remote partition". The actual
input values of the original call must be marshaled, e.g. as an output
stream. The output values and the result will be returned via an input
stream and deserialized from there into the actual parameters/result or
else into a remote exception occurrence to re-raise locally if that was
the outcome.

Here lie a lot of problems. First is non-portability of stream
attributes. Second is lack of support for bulky transfers and
multiplexing. It is highly desirable that the output stream could be
written in chunks as well as reading the input stream. E.g. if you pass
large objects or if you want to multiplex RPCs made from different tasks
rather than interlock them (which for synchronous RPC would result in
catastrophic performance).

The current Annex E is very crude to allow efficient, low-latency,
real-time implementations.

P.S. If Ada supported delegation, introspection and getter/setter
interface, then, probably, all remote call/objects stuff could be made
at the library level. But for now, compiler magic is needed.

[Randy Brukardt]

"Dmitry A. Kazakov" <mai...@dmitry-kazakov.de> wrote in message
news:rsfg32$1etf$1...@gioia.aioe.org...

> On 2020-12-29 00:41, Randy Brukardt wrote:
>> "Rod Kay" <roda...@gmail.com> wrote in message
>> news:30f2c0d8-58c2-4ca9...@googlegroups.com...
>>> Apologies for the prior empty post.
>>>
>>> Is it likely that the ARG might address the aformentioned issues ?
>>
>> As of now, it doesn't appear that there would be any point. Annex E is an
>> optional annex, and so far as we're aware, no compiler vendor has any
>> plans
>> for increasing support for that annex. So the ARG could change the annex
>> but
>> it seems unlikely that any changes would make it into implementations.
>> (We've been told not to expect even the implementation of bugs fixes
>> included in Ada 202x, even from the vendor that originally requested the
>> bug
>> fixes.)
>
> Why there should be any vendor support in the first place? Why not to
> redefine it as a set of abstract tagged types allowing custom user
> implementations like storage pools and streams do?

Marshalling/unmarshalling surely require vendor support, and there has to be
a standard interface for the marshalling stuff to talk to. That to me was
always the value of Annex E. My understanding is that there is not much
interest in doing any work at all, even to correct the mistakes in the
existing definitions.

In any case, Storage_Pools and Streams are some of the most expensive
features of Ada to support. That's not a model for "lightweight" support of
anything.

My advice would be to talk to your vendor if you feel strongly about this
sort of support.

Randy.

[Rod Kay]

Thanks to all for the info. Not quite the news I'd hoped for :) . I'll probably still persist with polyorb dsa for a while, since I've invested a goodish amount of time (and code) there.

Regards.

[Shark8]

On Thursday, December 24, 2020 at 6:30:59 AM UTC-7, Dmitry A. Kazakov wrote:
> On 2020-12-24 13:02, Maxim Reznik wrote:
> > I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
> > My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)
> >
> > https://github.com/reznikmm/garlic/tree/web_socket
> My question is how to proceed without GLADE/Garlic etc. I have DSA
> implemented, e.g. System.RPC. I need GNAT to accept it as such.

What do you mean by this?
How is GNAT "not accepting" it?
More, you mentioned the possibility of using a set of tagged-types, "like Storage_Pool", elsethread; is this how you implemented them, or a suggestion for a new standard interface?

>
> In a more distant perspective I need a work-around of stream attributes.
> They are non-portable, so there must an option to replace them for DSA
> and/or provide a non-stream parameter marshaling when the transport is a
> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For
> these you must map Ada types into the types defined by the protocol.

I have some ideas on how this could be achieved.
I'll make a note for my To-Do list on Byron, but I was thinking that a good method to achieve this for an OS (WRT ASN.1) would be to have at the base a SOM-like system with the base meta-object having a serialize/deserialize pair of methods (with one parameter being the encoding scheme)... I don't see why that [the idea, in principle] couldn't be 'lifted' up to the DSA level and integrated into it [PolyOrb or other DSA implementation].

[Dmitry A. Kazakov]

On 2021-01-11 15:59, Shark8 wrote:
> On Thursday, December 24, 2020 at 6:30:59 AM UTC-7, Dmitry A. Kazakov wrote:
>> On 2020-12-24 13:02, Maxim Reznik wrote:
>>> I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
>>> My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)
>>>
>>> https://github.com/reznikmm/garlic/tree/web_socket
>> My question is how to proceed without GLADE/Garlic etc. I have DSA
>> implemented, e.g. System.RPC. I need GNAT to accept it as such.
> What do you mean by this?
> How is GNAT "not accepting" it?

It does not invoke the provided DSA implementation for remote calls.

> More, you mentioned the possibility of using a set of tagged-types, "like Storage_Pool", elsethread; is this how you implemented them, or a suggestion for a new standard interface?

It was a suggestion. The current way is System.RPC see ARM E.5. It seems
that merely replacing it is not enough for GNAT.

>> In a more distant perspective I need a work-around of stream attributes.
>> They are non-portable, so there must an option to replace them for DSA
>> and/or provide a non-stream parameter marshaling when the transport is a
>> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For
>> these you must map Ada types into the types defined by the protocol.
> I have some ideas on how this could be achieved.
> I'll make a note for my To-Do list on Byron, but I was thinking that a good method to achieve this for an OS (WRT ASN.1) would be to have at the base a SOM-like system with the base meta-object having a serialize/deserialize pair of methods (with one parameter being the encoding scheme)...

The problem is that

1. They must doubly dispatch on the object type and on the container
type, since there could be any number of containers = encoding/decoding
schema.

2. They must support chunking. E.g. deserialize cannot be a function.
This will get you a whole bunch of nasty problems with partial
serialization and indefinite types. [I do not see these resolved without
properly done constructors and co-routines.]

[Shark8]

On Monday, January 11, 2021 at 8:32:25 AM UTC-7, Dmitry A. Kazakov wrote:
> On 2021-01-11 15:59, Shark8 wrote:
> > On Thursday, December 24, 2020 at 6:30:59 AM UTC-7, Dmitry A. Kazakov wrote:
> >> On 2020-12-24 13:02, Maxim Reznik wrote:
> >>> I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
> >>> My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)
> >>>
> >>> https://github.com/reznikmm/garlic/tree/web_socket
> >> My question is how to proceed without GLADE/Garlic etc. I have DSA
> >> implemented, e.g. System.RPC. I need GNAT to accept it as such.
> > What do you mean by this?
> > How is GNAT "not accepting" it?
> It does not invoke the provided DSA implementation for remote calls.

This is a dumb question, but did you remember to recompile the run-time library?
https://www.cs.fsu.edu/~baker/ada/gnat/html/gnat_ugn_20.html#SEC230
(I'm pretty sure you'd have to, to get a change into DSA behavior.)

> > More, you mentioned the possibility of using a set of tagged-types, "like Storage_Pool", elsethread; is this how you implemented them, or a suggestion for a new standard interface?
> It was a suggestion. The current way is System.RPC see ARM E.5. It seems
> that merely replacing it is not enough for GNAT.

Given "The package System.RPC is a language-defined interface to the PCS" it seems like that there are two readings:
(1) System.RPC defines the forms with which remote-calls are done, and/or
(2) implements the actual communications.
As it is written, I would think that #1 is the proper interpretation, if this is the case than recompiling System.RPC is "mucking about with the interface" and not much else.

The item "An object of the type Params_Stream_Type is used for identifying the particular remote subprogram that is being called, as well as marshalling and unmarshalling the parameters or result of a remote subprogram call, as part of sending them between partition" seems to indicate that #2 is the proper reading though.

> >> In a more distant perspective I need a work-around of stream attributes.
> >> They are non-portable, so there must an option to replace them for DSA
> >> and/or provide a non-stream parameter marshaling when the transport is a
> >> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For
> >> these you must map Ada types into the types defined by the protocol.
> > I have some ideas on how this could be achieved.
> > I'll make a note for my To-Do list on Byron, but I was thinking that a good method to achieve this for an OS (WRT ASN.1) would be to have at the base a SOM-like system with the base meta-object having a serialize/deserialize pair of methods (with one parameter being the encoding scheme)...
> The problem is that
>
> 1. They must doubly dispatch on the object type and on the container
> type, since there could be any number of containers = encoding/decoding
> schema.

Why?
You could have the encoding/decoding method as a parameter.

> 2. They must support chunking. E.g. deserialize cannot be a function.

Why?
I can't think of anything off the top of my head that directly prohibits us from making it a function; at least X'Stream'Input-wise.

> This will get you a whole bunch of nasty problems with partial
> serialization and indefinite types. [I do not see these resolved without
> properly done constructors and co-routines.]

I don't see why the partial-serialization is a problem: if you have some type's value to serialize, you're already constrained.
If you're reading from a stream to initialize/reconstruct a type-value, then you're doing X'Stream'Input, right?

[Dmitry A. Kazakov]

On 2021-01-11 21:35, Shark8 wrote:
> On Monday, January 11, 2021 at 8:32:25 AM UTC-7, Dmitry A. Kazakov wrote:
>> On 2021-01-11 15:59, Shark8 wrote:
>>> On Thursday, December 24, 2020 at 6:30:59 AM UTC-7, Dmitry A. Kazakov wrote:
>>>> On 2020-12-24 13:02, Maxim Reznik wrote:
>>>>> I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
>>>>> My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)
>>>>>
>>>>> https://github.com/reznikmm/garlic/tree/web_socket
>>>> My question is how to proceed without GLADE/Garlic etc. I have DSA
>>>> implemented, e.g. System.RPC. I need GNAT to accept it as such.
>>> What do you mean by this?
>>> How is GNAT "not accepting" it?
>> It does not invoke the provided DSA implementation for remote calls.
> This is a dumb question, but did you remember to recompile the run-time library?

No. Why should I?

> https://www.cs.fsu.edu/~baker/ada/gnat/html/gnat_ugn_20.html#SEC230
> (I'm pretty sure you'd have to, to get a change into DSA behavior.)

Do you mean that other implementations rebuild all RTL? That is a mess.
Is it possible to fool GNAT using linker instead of breaking everything?

>>> More, you mentioned the possibility of using a set of tagged-types, "like Storage_Pool", elsethread; is this how you implemented them, or a suggestion for a new standard interface?
>> It was a suggestion. The current way is System.RPC see ARM E.5. It seems
>> that merely replacing it is not enough for GNAT.
> Given "The package System.RPC is a language-defined interface to the PCS" it seems like that there are two readings:
> (1) System.RPC defines the forms with which remote-calls are done, and/or
> (2) implements the actual communications.
> As it is written, I would think that #1 is the proper interpretation, if this is the case than recompiling System.RPC is "mucking about with the interface" and not much else.

My understanding is that the Do_RPC gets called with the parameters of
the call written into the Params stream including the target function
name. Then you do communication with the remote host associated with the
Partition parameter. The communication basically reads the Params stream
and sends read stuff to the other side. Then you sit and wait for the
remote host to respond. If the response was success, you write its
payload into the Result stream. If the response indicates remote
exception you re-raise it locally.

BTW, there is a lot of overhead there. The design support streams being
actual communication streams. But for that, the partition would be set
into the stream prior calling to the first Write. Furthermore, it is to
tightly coupled. There is no indications where parameters start and end
in the stream. The remote host needs to be able to say "cut that, I do
not receive this call anyway", "I cannot parse parameters for this
function" etc.

>>>> In a more distant perspective I need a work-around of stream attributes.
>>>> They are non-portable, so there must an option to replace them for DSA
>>>> and/or provide a non-stream parameter marshaling when the transport is a
>>>> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For
>>>> these you must map Ada types into the types defined by the protocol.
>>> I have some ideas on how this could be achieved.
>>> I'll make a note for my To-Do list on Byron, but I was thinking that a good method to achieve this for an OS (WRT ASN.1) would be to have at the base a SOM-like system with the base meta-object having a serialize/deserialize pair of methods (with one parameter being the encoding scheme)...
>> The problem is that
>>
>> 1. They must doubly dispatch on the object type and on the container
>> type, since there could be any number of containers = encoding/decoding
>> schema.
> Why?
> You could have the encoding/decoding method as a parameter.

A scalar value is not enough. For one you need to enumerate all possible
methods to build a case statement on them, which you cannot do as
implementation must be located in different libraries.

Then you need some context to encode/decode referential types. Imagine a
parameter containing an access type, e.g. a node of a linked list. You
cannot deep-copy everything with an infinite recursion on top.

>> 2. They must support chunking. E.g. deserialize cannot be a function.
> Why?

Because it will block waiting all data read. So you will need an extra
task for each transfer in order to accumulate all chuncks.

>> This will get you a whole bunch of nasty problems with partial
>> serialization and indefinite types. [I do not see these resolved without
>> properly done constructors and co-routines.]
> I don't see why the partial-serialization is a problem: if you have some type's value to serialize, you're already constrained.
> If you're reading from a stream to initialize/reconstruct a type-value, then you're doing X'Stream'Input, right?

Remember, it does not block. So when you call X'Stream'Input you may get
0 stream elements read at any point. E.g. you read a 4-octet Integer.
You get only two octets, because the other two are not here. You cannot
return Integer, it is not yet constructed. You cannot wait, because you
are in the callback from the reader task. You must remember the state of
partial deserialization and pass control back to the reader. This is
what a co-routines are.