DSDL compiler signature issue

[OlliW]

hey Pavel,

I'm sorry to bother you again, this was supposed to be 5 min job but at the moment looks like a night mare.

I simply tried to get along with compiling DSDL definitions, mainly in order to get the signature and max bits for own definitions. As a start, I tried to generate the massively helpful list you provided to me here https://groups.google.com/forum/#!topic/uavcan/D5ZcsvirrbI.

I could generate something which looks very close, except that some of the signatures do NOT agree with those in your list !!!!!

I can't understand that, and am actually "scared". How is that possible.

I've put everything together in a .zip here: www.olliw.eu/drop/apm/forPavel.zip. It should allow you to exactly recreate what I've done. The final result is in dsdlc-standard.out.txt.cvt.

In short, I've synced with the uavcan git yesterday, and took the dsdl and dsdl_compiler pieces in my own directory. To make the compiler usable I simply copied libuavcan_dsdlc to libuavcan_dsdlc.py. dsdlc-standard.bat compiles the standard dsdl definitions, and converts the output into something more readable.

Thy, Olli

[Pavel Kirienko]

Hey, Olli!

No reason to be alarmed here. You've just mixed up the concepts of DSDL Signature and Data Type Signature. Please read up on this here: http://uavcan.org/Specification/3._Data_structure_description_language/#data-type-compatibility. Note that we're currently working on improving the specification, you can follow our progress and make suggestions here: https://github.com/UAVCAN/uavcan.github.io/pulls.

Also, I couldn't help but notice how peculiar your approach at handling the data types is. You are scraping the logs of a compiler in order to extract the data type information from them (observe that this is precisely what caused the confusion: the compiler logs DSDL Signatures, but you need Data Type Signatures!). Instead of that, I suggest you to just use the pyuavcan's parser directly from your Python script. Here's how: http://uavcan.org/Implementations/Pyuavcan/Tutorials/4._Parsing_DSDL_definitions/; a more practical example: https://github.com/UAVCAN/libcanard/blob/master/show_data_type_info.py.

Hope this helps!

Pavel.

--
You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/8fd5db6b-602a-47a4-8ee3-b429a9bf10d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[OlliW]

Hey Pavel,

first thx a LOT for the quick, and clarifying, response.
indeed, I have mixed that up. In fact, I totally forgot about that other signature.
It's amazing that for many data types both signatures agree (which certainly contributed to me not realizing the difference). I guess I just don't understand the concept, but I guess that's ok. :)

As regards the pecularity of my approach: Please recall that I'm using libcanard and just need this little piece of info. I'm not using libuavcan, and this is largely because libuavcan to me is an incomprehensible monster I really don't want to touch. Unfortunately, there is no standalone compiler one just could use. I have seen the link you mentioned but when reading it I thought "oh nooo, seriously? just to get this info?". Gladly, there is also this other much simpler approach, so I did that. When looking at the generated hpps I was expecting to find #defines on top of them with all the information I was after, but no-no, not so easy. The max lengths and signatures are somewhere deep in the code. I then saw that -v gives seemingly exactly the same information. I really didn't had it on record that -v might output info which is not relevant for the hpp files. So, if you find the taken approach peculiar, then, honestly, it is a direct result of the fact that a simple thing has successfully been made not so simple. :)

Anyway, I think the lesson I should take is that correct signatures are those which I find in the .hpp files (at least it says ::uavcan::DataTypeSignature).

So I should parse them. It's probably even simpler than what I do now.

Thx again for your patience and help,
Olli

[OlliW]

no, that's not it

I just looked at the signatures stored in the hpp files. For exampe for protocol.param.GetSet it says

inline ::uavcan::DataTypeSignature GetSet_::getDataTypeSignature()
{
    ::uavcan::DataTypeSignature signature(0xB7D14152F13221EDULL);
:
:
   return signature;
}

Here I everywhere just read "DataTypeSignature" and not "DSDLSignature", which makes me believe it's the Data Type Signature and not the DSDL Signature. Yet, when I look up your table I find

Full Data Type Name                                  | DDTID |   Type Signature   |  Max Bit Len  

--------------------------------------------------------------------------------------------------

:
:

uavcan.protocol.param.GetSet                              11   0xa7b622f939d1a4d5    1791 / 2967
:

which obviously is a different signature, even though it's named "Type Signature" which I would read as to mean "Data Type Signature".

???

[Pavel Kirienko]

Olli,

> So, if you find the taken approach peculiar, then, honestly, it is a direct result of the fact that a simple thing has successfully been made not so simple.

Your approach is peculiar because instead of using a dedicated tool that does exactly what you need, you chose to misuse a compiler that has absolutely no relation to the library you're using. The tool that you need is this one: https://github.com/UAVCAN/libcanard/blob/master/show_data_type_info.py. Just run the script and you're all set. No coding/parsing involved.

> Here I everywhere just read "DataTypeSignature" and not "DSDLSignature", which makes me believe it's the Data Type Signature and not the DSDL Signature. Yet, when I look up your table I find

Please stop looking at libuavcan, it's not what you need right now. The code you're referring to computes the data type signature at the run time. You can't find the data type signature anywhere in the generated code, it simply doesn't exist until the application is running, it is explained in the documentation.

Please just use the linked tool, that's all you need right now.

Pavel.

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/e58094ff-a323-48cc-9cd2-b3f7702730df%40googlegroups.com.

[OlliW]

Hey Pavel,

thx again

I gave the tool you mentioned a quick try, but failed, I'll dig into it.

I never would have suspected that the DataTypes in the .hpp are not the data types and that these are computed at run time. I'm soo glad you've created libcanard. And I'm so glad that you're so supportive. Thx, sir.

Oh man, I just wanted to get the signature for a home-brewed message ... LOL.

Olli

[OlliW]

Hey Pavel

I got the suggested code show_data_type_info.py running. It indeed provides the info one needs.

I'm somewhat sorry, since I definitely have failed to realize the existence of that code, so, all my fault.

Maybe this suggestion: A point with that code I do not like that much is that it requires the standard DSDL data types to be present in the folder dsdl_files, and otherwise fails to run (very inconvenient for non git users), and outputs them all too.
I have noted in the __init__.py for the uavcan package that it is possible to set an arg 'exclude_dist' to bypass this, but
* it seems not possible to set this arg as external user
* even if set, the code fails, since e.g. dynamic_node_id.py is included, and this requires uavcan.protocol.dynamic_node_id.Allocation() which is of course not present with 'exclude_dist' (even if the folder dsdl_files is populated)
The later could probably qualify as bug.

Many Thx, Olli

[Pavel Kirienko]

Hey Olli,

You can't take apart a software package and expect it to keep functioning properly.

Perhaps the first time you tried to run the script, it exited with an error saying this:

PyUAVCAN is not installed. Please install from PIP: sudo pip3 install uavcan

What that means is that you should install the required dependency, which is the PyUAVCAN package, by running the specified command. Copying the code manually from the repository may work, but only if you really know why you're doing that and if you have an extra hour to kill.

If you're on Windows, then the proper command may look like this: python -m pip install "uavcan>=1.0.0.dev27"

Hope this helps.

Pavel.

--
You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/73a362e4-2dbc-4052-b1be-6f5c2e5ae253%40googlegroups.com.

[OlliW]

well, I understand this install thing

I have to disappoint you however that things don't seem to work exactly as you seem to have intended them to work
E.g., I have seen from the code that the intention is to check for uavcan being installed and put out the warning you were mentioning, but no, so far I never have seen that message.
I just redid this on another computer, using exactly the line you gave me, and it told me "Successfully installed monotonic-1.3 uavcan-1.0.0-dev27", and yet it remains so that the dsdls need to be present in the folder dsdl_files. I still don't see how to set the arg 'exclude_dist' as user, it certainly can't be give as command line parameter (and this seems clear from the code). And it still remains that then I set it by hand one gets the error I've mentioned in the 2nd bullet (and to me this also seems clear from the code).
So, I'm not at all convinced that what I'm describing is just due to me not installing it. :)

I just discovered that then I place my own dsdls also in "c:\users\olli\uavcan_vendor_specific_types\" that it will then take them also into account automatically. How annoying is this.

Anyway, this had been just a suggestion, to maybe offer an option which makes exclude_dist working for average users. Not that important.


I do have found btw a much more "wired" issue. I've created my own broadcast message, stored in storm32/8300.Status.uavcan

# Gimbal Attitude, angles in radians +-pi, in gimbal frame.
float32 pitch
float32 roll
float32 yaw

now, if I place it in the same directory where show_data_type_info resides into a subfolder "mydsdl\storm32\8300.Status.uavcan" and compile it with the option -c mydsdl I do get as output
mydsdl.storm32.Status                                   8300   0x3233511cbe74e06f      96

however, if I place that same file into "c:\users\olli\uavcan_vendor_specific_types\storm32\8300.Status.uavcan", when I do get the output
storm32.Status                                          8300   0x6f6fe9ce9d69c431      96

It's the very same file, just placed at different locations (not in both at the same time), but compiled from the same cmd shell from the same folder with the same file show_data_type_info.py, only that in one case I give the parameter -c mydsdl and in the other not.

???

No, sir, I can't make any sense of this sort of behavior. (just took me a couple of hours now since the first signature produces crc errors)

:)

cheers, Olli


PS: This uavcan thing is really pulling my hairs, I never had this level of difficulties before with anything I did, not even close. I mean, programming is a straightforward thing. Since ArduCopter's Uavcan support is also still in its infancy, I guess I really should never have gotten my self into this.

[Pavel Kirienko]

> E.g., I have seen from the code that the intention is to check for uavcan being installed and put out the warning you were mentioning, but no, so far I never have seen that message.

Perhaps this is because you have a local (incomplete) copy of PyUAVCAN next to the script you're trying to run, so the script tries to import that instead of the proper package. Please try that in a clean environment, e.g. just check out libcanard and try to run the script there.

> I do have found btw a much more "wired" issue. I've created my own broadcast message, stored in storm32/8300.Status.uavcan

> mydsdl.storm32.Status                                   8300   0x3233511cbe74e06f      96

> storm32.Status                                          8300   0x6f6fe9ce9d69c431      96

Please observe that the namespace names are different. This accounts for the difference in signatures.

Pavel.

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/4febf690-6151-4ed3-9a15-6b9c18726f6f%40googlegroups.com.