UAVCAN for Hobbiests II

[OlliW]

Hey Pavel,

I have now two nodes running and exchanging NodeStatus and GetNodeInfo, as well as have build&written a SLCAN adapter, and your GUI Tool is working great (I just don't like that startup procedure, with this extra window, not very Window-ish LOL)(I hope that the fact that I've created a cheap DIY SLCAN adapter won't interfere with your Babel sales). So, I'm prepared to do some real meat, my first attempt will be to do a magnetic field sensor (which had actually been the driving goal). Thx for all the work you put into all that, and your help. :) 

There is one question though I could not resolve: GetNodeInfo has two variable fields, uint8[<=255] certificate_of_authenticity in the HardwareVersion field, and uint8[<=80] name. The resulting ambiguity in the payload length can however not be resolved from the payload length itself. So, question is: How, if a CA is present is one supposed to determine the lengths of these two fields?

Maybe a suggestion for the documentation: I would find it extremely helpful if e.g. the data type signatures were also given in the docu, i.e. on the List of standard data types page. Should be a relatively trivial modification to the parser you probably use (I love Python, but it's portability to different PCs is a mess).

Thx,
Olli

[Pavel Kirienko]

Hey Olli,

> GetNodeInfo has two variable fields, uint8[<=255] certificate_of_authenticity in the HardwareVersion field, and uint8[<=80] name. The resulting ambiguity in the payload length can however not be resolved from the payload length itself. So, question is: How, if a CA is present is one supposed to determine the lengths of these two fields?

Dynamic arrays are prefixed with length fields. You can read up on this here: http://uavcan.org/Specification/3._Data_structure_description_language/#data-serialization

> I would find it extremely helpful if e.g. the data type signatures were also given in the docu, i.e. on the List of standard data types page.

Sadly, at the moment this can't be done. The website uavcan.org is hosted on GitHub Pages, which is a static only hosting with a ton of restrictions. However, I can help you here with this:

Full Data Type Name                                  | DDTID |   Type Signature   |  Max Bit Len  

--------------------------------------------------------------------------------------------------

uavcan.CoarseOrientation                                 N/A   0x271ba10b0dac9e52      16

uavcan.Timestamp                                         N/A   0x05bd0b5c81087e0d      56

uavcan.equipment.actuator.ArrayCommand                  1010   0xd8a7486238ec3af3     484

uavcan.equipment.actuator.Command                        N/A   0x8d9a6a920c1d616c      32

uavcan.equipment.actuator.Status                        1011   0x5e9bba44faf1ea04      64

uavcan.equipment.ahrs.MagneticFieldStrength             1001   0xe2a7d4a9460bc2f2     196

uavcan.equipment.ahrs.Solution                          1000   0x72a63a3c6f41fa9b     668

uavcan.equipment.air_data.AngleOfAttack                 1025   0xd5513c3f7afac74e      40

uavcan.equipment.air_data.IndicatedAirspeed             1021   0x0a1892d72ab8945f      32

uavcan.equipment.air_data.Sideslip                      1026   0x7b48e55fcff42a57      32

uavcan.equipment.air_data.StaticPressure                1028   0xcdc7c43412bdc89a      48

uavcan.equipment.air_data.StaticTemperature             1029   0x49272a6477d96271      32

uavcan.equipment.air_data.TrueAirspeed                  1020   0x306f69e0a591afaa      32

uavcan.equipment.camera_gimbal.AngularCommand           1040   0x4af6e57b2b2be29c      80

uavcan.equipment.camera_gimbal.GEOPOICommand            1041   0x9371428a92f01fd6     104

uavcan.equipment.camera_gimbal.Mode                      N/A   0x9108c7785aeb69c4       8

uavcan.equipment.camera_gimbal.Status                   1044   0xb9f127865be0d61e     228

uavcan.equipment.device.Temperature                     1110   0x70261c28a94144c6      40

uavcan.equipment.esc.RPMCommand                         1031   0xce0f9f621cf7e70b     365

uavcan.equipment.esc.RawCommand                         1030   0x217f5c87d7ec951d     285

uavcan.equipment.esc.Status                             1034   0xa9af28aea2fbb254     110

uavcan.equipment.gnss.Auxiliary                         1061   0x9be8bdc4c3dbbfd2     125

uavcan.equipment.gnss.ECEFPositionVelocity               N/A   0x24a5da4abee3a248     792

uavcan.equipment.gnss.Fix                               1060   0x54c1572b9e07f297     628

uavcan.equipment.gnss.Fix2                              1063   0xca41e7000f37435f    1769

uavcan.equipment.gnss.RTCMStream                        1062   0x1f56030ecb171501    1040

uavcan.equipment.hardpoint.Command                      1070   0xa1a036268b0c3455      24

uavcan.equipment.hardpoint.Status                       1071   0x624a519d42553d82      56

uavcan.equipment.indication.BeepCommand                 1080   0xbe9ea9fec2b15d52      32

uavcan.equipment.indication.LightsCommand               1081   0x2031d93c8bdd1ec4     485

uavcan.equipment.indication.RGB565                       N/A   0x58a7cef41951ec34      16

uavcan.equipment.indication.SingleLightCommand           N/A   0xe894b8b589807007      24

uavcan.equipment.power.BatteryInfo                      1092   0x249c26548a711966     437

uavcan.equipment.power.CircuitStatus                    1091   0x8313d33d0ddda115      56

uavcan.equipment.power.PrimaryPowerSupplyStatus         1090   0xbba05074ad757480      47

uavcan.equipment.range_sensor.Measurement               1050   0x68fffe70fc771952     120

uavcan.equipment.safety.ArmingStatus                    1100   0x8700f375556a8003       8

uavcan.protocol.AccessCommandShell                         6   0x59276b5921c9246e    1040 / 2097

uavcan.protocol.CANIfaceStats                            N/A   0x13b106f0c44ca350     144

uavcan.protocol.DataTypeKind                             N/A   0x9420a73e008e5930       8

uavcan.protocol.GetDataTypeInfo                            2   0x1b283338a7bed2d8     671 / 743

uavcan.protocol.GetNodeInfo                                1   0xee468a8121c46a9e       0 / 3015

uavcan.protocol.GetTransportStats                          4   0xbe6f76a7ec312b04       0 / 578

uavcan.protocol.GlobalTimeSync                             4   0x20271116a793c2db      56

uavcan.protocol.HardwareVersion                          N/A   0x0ad5c4c933f4a0c4    2192

uavcan.protocol.NodeStatus                               341   0x0f0868d0c1a7c6f1      56

uavcan.protocol.Panic                                      5   0x8b79b4101811c1d7      59

uavcan.protocol.RestartNode                                5   0x569e05394a3017f0      40 / 1

uavcan.protocol.SoftwareVersion                          N/A   0xdd46fd376527fea1     120

uavcan.protocol.debug.KeyValue                         16370   0xe02f25d6e0c98ae0     502

uavcan.protocol.debug.LogLevel                           N/A   0x711bf141af572346       3

uavcan.protocol.debug.LogMessage                       16383   0xd654a48e0c049d75     983

uavcan.protocol.dynamic_node_id.Allocation                 1   0x0b2a812620a11d40     141

uavcan.protocol.dynamic_node_id.server.AppendEntries      30   0x8032c7097b48a3cc     249 / 33

uavcan.protocol.dynamic_node_id.server.Discovery         390   0x821ae2f525f69f21      51

uavcan.protocol.dynamic_node_id.server.Entry             N/A   0x7faa779d64fa75c2     168

uavcan.protocol.dynamic_node_id.server.RequestVote        31   0xcdde07bb89a56356      72 / 33

uavcan.protocol.enumeration.Begin                         15   0x196ae06426a3b5d8     759 / 8

uavcan.protocol.enumeration.Indication                   380   0x884cb63050a84f35     815

uavcan.protocol.file.BeginFirmwareUpdate                  40   0xb7d725df72724126    1616 / 1031

uavcan.protocol.file.Delete                               47   0x78648c99170b47aa    1608 / 16

uavcan.protocol.file.EntryType                           N/A   0x6924572fbb2086e5       8

uavcan.protocol.file.Error                               N/A   0xa83071ffea4fae15      16

uavcan.protocol.file.GetDirectoryEntryInfo                46   0x8c46e8ab568bda79    1640 / 1632

uavcan.protocol.file.GetInfo                              45   0x5004891ee8a27531    1608 / 64

uavcan.protocol.file.Path                                N/A   0x12aefc50878a43e2    1608

uavcan.protocol.file.Read                                 48   0x8dcdca939f33f678    1648 / 2073

uavcan.protocol.file.Write                                49   0x515aa1dc77e58429    3192 / 16

uavcan.protocol.param.Empty                              N/A   0x6c4d0e8ef37361df       0

uavcan.protocol.param.ExecuteOpcode                       10   0x3b131ac5eb69d2cd      56 / 49   

uavcan.protocol.param.GetSet                              11   0xa7b622f939d1a4d5    1791 / 2967 

uavcan.protocol.param.NumericValue                       N/A   0x0da6d6fea22e3587      66

uavcan.protocol.param.Value                              N/A   0x29f14bf484727267    1035

Pavel.

--
You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/bd700165-9cb2-4132-a816-be1e88983cb5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[OlliW]

dammed, I was kind of sure (or should I say afraid?) that you're going to point me to somewhere in the docu ... by now I really spend so much time on it, but there are still gems to be discovered :)

I'm swept of my feet ... thx soo much for this list.

I've just seen that ArduCopter yesterday had so many UAVCAN related commits ... I think, and it's highly overdue, UAVCAN will finally start to fly high.

All the best, Olli

[OlliW]

Hey Pavel

I've got now also the parameters working (this was a bit daunting, the bit stream for the GetSet messages and the use of the Empty isn't immediately obvious :)). I think the next step will be GPS.

Question: It is suggested to also support uavcan.protocol.RestartNode.It is not clear to me if after the restart the node is supposed to "forget" its ID in case it was allocated dynamically, such that the allocation needs to be repeated, or to keep it but just so to say restart everything else ?

Comment: The procedure of finding the parameters relies on the fact that the index values are continues from 0 up to a maximum (I believe). I think this could be added to the documentation. Also, it is not absolutely clear what indicates that a parameter is not existing, empty value, empty string, or both. The docu seems to say different things.

Thx, Olli

[OlliW]

I wanted to add, the above list you offered is invaluable ! THX !

[Pavel Kirienko]

HI Olli,

 

> It is suggested to also support uavcan.protocol.RestartNode.It is not clear to me if after the restart the node is supposed to "forget" its ID in case it was allocated dynamically, such that the allocation needs to be repeated, or to keep it but just so to say restart everything else ?

 

The node is supposed to allocate anew upon boot. This is now explicitly stated in the spec, thanks for pointing this out.

 

> The procedure of finding the parameters relies on the fact that the index values are continues from 0 up to a maximum (I believe). I think this could be added to the documentation.

 

Done.

 

> Also, it is not absolutely clear what indicates that a parameter is not existing, empty value, empty string, or both. The docu seems to say different things.

 

Clarifications added.

 

Thanks,

Pavel.

-- 

Pavel Kirienko

+7 926 253 2081

Zubax Robotics OÜ

25.04.2017, 03:36, "OlliW" <oll...@gmail.com>:

I wanted to add, the above list you offered is invaluable ! THX !

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.

To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+un...@googlegroups.com.

To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/23dc8d2e-9bc2-4964-b65b-e0488223e7f6%40googlegroups.com.

[OlliW]

Hey Pavel

many thx for these clarifications

I think I'm doing the right thing, but I still feel a bit uncomfortable with uavcan when it comes to it's many edge cases, so, may be you could confirm/disconfirm:

In order to process the RestartNode I should wait until all transmissions are completed before restarting the node (I think). I think I thus need to wait for the TxQueue to be empty, and then for the CAN mailboxes to be empty, i.e. I'm doing something like

static const uint32_t AllTME = CANARD_STM32_CAN_TSR_TME0 | CANARD_STM32_CAN_TSR_TME1 | CANARD_STM32_CAN_TSR_TME2;

while( canardPeekTxQueue(&ins)!= NULL ){};

while( (BXCAN->TSR & AllTME) != AllTME ){};

ReBoot();

Is this fully correct, or do I miss an edge case?

I think I'll find it convenient to add a canardSTM32TransmitDone() or similarly named routine to the STM32 driver.

Cheers, Olli

[Pavel Kirienko]

Hi Olli,

 

> In order to process the RestartNode I should wait until all transmissions are completed before restarting the node 

 

It is not strictly required, but yes, it is recommended to ensure that the node leaves no unterminated transfers when it goes down.

 

Your code sample looks okay, except that your first loop will never end unless you flush the queue within its body. The peek operation is non-mutating (read the documentation in canard.h for details).

 

> I think I'll find it convenient to add a canardSTM32TransmitDone() or similarly named routine to the STM32 driver.

 

Perhaps.

 

-- 

Pavel Kirienko

+7 926 253 2081

Zubax Robotics OÜ

 

 

 

25.04.2017, 14:06, "OlliW" <oll...@gmail.com>:

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+un...@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/59de056e-c5d4-47bd-aae0-6be13ed4fd1e%40googlegroups.com.

[OlliW]

Hey Pavel

 

except that your first loop will never end unless you flush the queue within its body

ha, yes, of course ... my actual code somewhat different, especially non blocking ... many thx for pointing this out however

It seems I'm on the right track though, that's relieving to know. T H X.

I'm actually starting to wonder how much resources this uavcan thing takes ... I guess I should somehow measure that ... I start to like the idea of having the STorM32 supporting uavcan, if that's possible without having to switch to another mcu (the F103 is already quite busy, and this has not become better with recent developments like going encoders or gps tracking LOL)

In that context: I wasn't able yet to understand the decode and encode routines, but my first impulse is that they are not most efficient. To me it looks as if these routines always start from the beginning over and over again (and that the load thus increases superlinearly if not exponentially with the length of a message). I would think, or could imagine, that a first - next approach, where results of the previous step are kept in a structure (along the lines of the DOS filefind, or the strtok thing, etc.) could be significantly more efficient. Would you agree with that, and could this be a potentially worthwhile approach, or would you think that the performance gain would be marginal?

Thx so much again,
Olli

[Pavel Kirienko]

Hi Olli,

 

The codec routines do have some room for optimization. A pull request improving their implementation would be welcome. However, I don't foresee any significant improvements in performance, yet the ROM footprint might suffer. Someone should volunteer to research this issue in a more detailed way.

-- 

Pavel Kirienko

+7 926 253 2081

Zubax Robotics OÜ

25.04.2017, 17:30, "OlliW" <oll...@gmail.com>:

--
You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+un...@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/ed73ea35-ebc9-4204-8ec4-0f73c11fbbcd%40googlegroups.com.

[OlliW]

Hey Pavel

 

I don't foresee any significant improvements in performance

that's sad
 

 Someone should volunteer to research this issue in a more detailed way.

it might happen
 
two minor issues with the GUI Tool (which is great, in case I haven't said it yet):
* after an Restart, the parameter list sticks to its old values, but they might have changed due to the restart (e.g. Erase All -> Restart). The list should probably be cleared.
* when the GUI Tool is running, but the slcan adapter e.g. switched off or disconnected, then it seems there is no way to recover except to close and open the GUI Tool again, such as e.g. a reconnect button, or whatever.

I think my mag node is working quite ok now, and I should be close to a real world test.

All the best, Olli

[Pavel Kirienko]

Thanks Olli,

 

These issues were reported to the GUI Tool repo.

-- 

Pavel Kirienko

+7 926 253 2081

Zubax Robotics OÜ

26.04.2017, 02:07, "OlliW" <oll...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/57bde681-ec62-4934-b762-84f38d3d834d%40googlegroups.com.

[OlliW]

Hey Pavel

I'm making good progress with the uc4h node, I've moved forward to also use the gps. I think everything is working, I only have some issues/question with setting the fields of gnss.fix and gnss.auxiliary.

I use gnss.fix and not gnss.fix2, becasue this seems to be what is (only) supported by ArduPilot.

In order to get the data from the gps (m8), I made it send the NAV-PVT and NAV-DOP ubx messages (and only those so far). Again, from peeking into ArduPilot it seems to me that this two are the only ones which are actually used (for NAV-PVT enabled gpses). (it sues some more, but these look to me are only for "statistics" or other uses).

gnss.fix:
* I did get it working correctly, however I would have estimated that I need 4 bit more then no cov matrices are used. It's working as said, but I don't understand why the len field is only 4 bits and not 8 bits. These variable len rules are really difficult. Honestly, I think, they could have been simplifer, with nearly zero performance loss (we're talking about saving few bits in 200-400 bit messages).
 
* I have yet to learn the differnt time scales, but I guess: It is OK for proper operation to set all time fields (timestamp,gnss_timestamp,gnss_time_standard,num_leap_seconds) to zero, right?

* I'm not sure how to set the cov matrices, if I wanted to use them. I guess, fundamentally, with the data available from NAV-PVT, they are diagonal matrices with the diagonals equal and set to ... yes, what exactly? (vAcc^2, vAcc^2, hAcc^2) for the position and (sAcc^2, sAcc^2, sAcc^2) for the speed (times the proper scaling factors)?

* cov matrices II: All non-diagonals are zero, do I have to transmit them, or would it be OK to send just the 3 diagonal values, and the other side would understand that (i.e. expand that properly into a matrix)? Would it be then OK for the velocity cov matrix, to only send one value (as the diagonals are equal)? I think this is what is suggested in http://uavcan.org/Specification/5._Application_level_conventions/#matrix-representation.

gnss.auxiliary:
* There should I get sats_visible from? It doesn't seem to be used by ArduPilot. Could I just set it equal to the used sats?

Thx, Olli

[Pavel Kirienko]

Hi Olli,

> I use gnss.fix and not gnss.fix2, becasue this seems to be what is (only) supported by ArduPilot.

APM will support Fix2 pretty soon, if not already. Eugene had the intention of implementing this feature.

> I did get it working correctly, however I would have estimated that I need 4 bit more then no cov matrices are used. It's working as said, but I don't understand why the len field is only 4 bits and not 8 bits. These variable len rules are really difficult. Honestly, I think, they could have been simplifer, with nearly zero performance loss (we're talking about saving few bits in 200-400 bit messages).

You can refer to libuavcan or pyuavcan to see how to handle variable length fields properly. In the specific case of gnss.Fix, the length field of the last array is omitted because it is the last variable length field of the message. See "tail array optimization".

> It is OK for proper operation to set all time fields (timestamp,gnss_timestamp,gnss_time_standard,num_leap_seconds) to zero, right?

You can only set zero the fields "timestamp" and "num_leap_seconds". The other two must be populated properly as follows:

- gnss_timestamp - GNSS time provided by the receiver.

- gnss_time_standard - according to the time base reported by the receiver (usually UTC or GPS time).

> I'm not sure how to set the cov matrices

Refer to Zubax GNSS: https://github.com/Zubax/zubax_gnss/blob/master/firmware/src/board/ublox.cpp#L361-L369

> cov matrices II: All non-diagonals are zero, do I have to transmit them, or would it be OK to send just the 3 diagonal values, and the other side would understand that (i.e. expand that properly into a matrix)? Would it be then OK for the velocity cov matrix, to only send one value (as the diagonals are equal)?

Yes to both questions.

> There should I get sats_visible from? It doesn't seem to be used by ArduPilot. Could I just set it equal to the used sats?

You can, but you shouldn't do that. The number of visible satellites is provided by some other u-blox message. You can refer to the sources of Zubax GNSS for an example.

Pavel.

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.

To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.

To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/cc61b11a-4f91-4ead-a784-fbbf1c5869be%40googlegroups.com.

[OlliW]

Hey Pavel

thx so much again

You can refer to libuavcan or pyuavcan to see how to handle variable length fields properly. In the specific case of gnss.Fix, the length field of the last array is omitted because it is the last variable length field of the message. See "tail array optimization".

I don't get it. I'm missing 4 bits. In "finally deciding" on the length fields I let me guide by the list in the above. My count is this (for a max populated gnss fix):

timestamp                         56
gnss_timestamp                56
gnss_time_standard           3
void5                                 5   
num_leap_seconds            8
longitude_deg_1e8             37
latitude_deg_1e8               37
height_ellipsoid_mm          27
height_msl_mm                 27
ned_velocity                      48
sats_used                         6
status                               2
pdop                                16
void4                                4
                                       -----
                   sum so far = 332
len_pos_cov                     4   
position_covariance          144
velocity_covariance          144
                                      -----
                     sum total = 624       

In your list which you thankfully have given I find however a maximum value of 628 !!
And this was what made me believe that in this case the tail array optimization, for some reasons I don't understand, doesn't bite. Because this would add another 4 bits for the length of the vel-cov, adding up to 628.

???

Cheers, Olli

 

[OlliW]

sorry, I fogot:

You can only set zero the fields "timestamp" and "num_leap_seconds". The other two must be populated properly as follows:

- gnss_timestamp - GNSS time provided by the receiver.

- gnss_time_standard - according to the time base reported by the receiver (usually UTC or GPS time).

That's not what is said in the docu: 

#
# Time solution.
# Time standard (GPS, UTC, TAI, etc) is defined in the field below.
#
uavcan.Timestamp gnss_timestamp    # GNSS timestamp, if available, otherwise zero

#
# Time standard used in the GNSS timestamp field.
#
uint3 GNSS_TIME_STANDARD_NONE = 0  # Time is unknown
uint3 GNSS_TIME_STANDARD_TAI  = 1
uint3 GNSS_TIME_STANDARD_UTC  = 2
uint3 GNSS_TIME_STANDARD_GPS  = 3
uint3 gnss_time_standard

Maybe you could update that. :)

Cheers, Olli


[Pavel Kirienko]

Thanks Olli.

Regarding the missing 4-th bit, the maximum size of the data structure is provided for the worst case, that is, assuming that the tail array optimization would not be possible. For example, if there is an outer data type that carries an array of gnss.Fix, the tail array optimization will not be performed, and so each data structure will be at most 628 bits. In a standalone (top-level) message, TAO can be done, therefore we get 624 bits.

Regarding the inconsistency in the documentation, thank you, I'm fixing it right away.

Pavel.

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/3e3aa359-123d-41b9-a35c-5f2e994f1edd%40googlegroups.com.

[OlliW]

many thx for the clarification

but, honestly, this is most confusing ...
in addition, since the code which generates all that must know if TAO applies or not (otherwise it would parse the DSDLs into wrong code) it's not clear why it would not adjust the max length accordingly
(IMHO the "worst case" argument doesn't apply because it's not a worst case, along the same lines you should when argue that a void5 should be taken as 8bits as worst case ... ;))


On another matter, I think you have to properly define what in gnss.fix the gnss_timestamp is supposed to be.

I've spend now one day researching the time bases, and it is pretty clear that there is no such thing as a UTC timestamp ... the fact that a maybe large community got used to certain habits are no justification for misusing international convention and/or brewing up own definitions in their name. I have a very clear opinion on that :D.  (which I'm kind of convinced you will reject immediately, but I think it's clear I'm right LOL, especially when it comes to setting a "definition" or "standard").

I think Unix time and UTC must be distinguished. Moreover, when converting UTC to a timestamp the procedure should be explicitly defined, and not implicitly by whatever habits of whatever communities. This includes the handling of leap seconds.

I could propose this (as example):

GNSS_TIME_STANDARD_UTC  = 2

The timestamp is derived from the current UTC time minus UTC 1.1.1970. Leap seconds are taken into account. For convenience, the field leap_seconds reports the number of leap seconds with respect to Unix time, if the value is non zero.

GNSS_TIME_STANDARD_UNIX = 4

The timestamp corresponds to Unix time. Leap seconds are not taken into account. For convenience, the field leap_seconds reports the number of leap seconds between UTC and Unix time, if the value is non zero.

GNSS_TIME_STANDARD_GPS = 3

For convenience, the field leap_seconds reports the number of leap seconds between GPS time and Unix time, if the value is non zero.

For GNSS_TIME_STANDARD_GPS I would have appreciated a clear definition. I assume that it is supposed to be
week * (7*24*60*60*1000000) + iTOW * 1000 + fTOW / 1000;


I also note inconsistencies of how the fix and gnss timestamp are handled when (i) a zubax gps is connected or (ii) a m8 gps is connected to the pixhawk. These are related to how the NAV-PVT's valid and flags fields are handled or not handled. But I guess I might raise that topic better at ArduPilot discuss.

Cheers, and thx again for all,
Olli

[Pavel Kirienko]

Hi Olli,

I really appreciate your feedback.

> in addition, since the code which generates all that must know if TAO applies or not (otherwise it would parse the DSDLs into wrong code) it's not clear why it would not adjust the max length accordingly

> (IMHO the "worst case" argument doesn't apply because it's not a worst case, along the same lines you should when argue that a void5 should be taken as 8bits as worst case ... ;))

The code that generates DSDL serialization code is not in the right context to know whether TAO applies or not, this is resolved at run time. "Worst case" is a perfectly valid term here, since it is meant to indicate that the worst case is the case where TAO cannot be applied to remove the array size field.

> I've spend now one day researching the time bases, and it is pretty clear that there is no such thing as a UTC timestamp ...

This is a very valid observation, thank you. We should amend the specification with correct definitions of the time bases in use. It is true that what we call "UTC timestamp" is, in fact, the UTC time minus 1970-01-01-00:00:00 minus leap seconds, as you correctly stated. The "GPS timestamp" is implicitly defined likewise, and the same goes for TAI.

The definition of the message gnss.Fix2 should be amended with the proper definition of the time bases in use. I don't think that this warrants changes to the actual definitions. If you could submit a pull request, that would be most appreciated (or at least publish a set of strict definitions here, for me to commit it later). Thanks in advance!

Concerning the leap seconds: unlike the above, this term is quite well defined and we must not introduce any special cases like this:

> For convenience, the field leap_seconds reports the number of leap seconds with respect to Unix time, if the value is non zero.

> For convenience, the field leap_seconds reports the number of leap seconds between UTC and Unix time, if the value is non zero.

> For convenience, the field leap_seconds reports the number of leap seconds between GPS time and Unix time, if the value is non zero.

Such amendments would serve no purpose, since the concept of a "leap second" is quite well understood. See https://en.wikipedia.org/wiki/Leap_second#Insertion_of_leap_seconds. As of today, the number of leap seconds is 27, and it is not dependent on any particular time base, but rather on the accumulated drift of the angular rate of our planet.

Thanks,

Pavel.

--
You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/32ec14cd-9626-48e4-994f-304803b9483e%40googlegroups.com.

[OlliW]

Hey Pavel

as regards leap seconds I agree. My attempts for "improved" descriptions were admittedly not fully worded out. What I aimed for was to specify what the value in leap seconds is supposed to mean. One could (!) let them have different values in the various cases. E.g. in the case of GPS time, since it started later, the number of leap seconds for converting to Unix is different, and one could give back this corrected number, so that users could easily convert without having to have the difference in mind. The terminus "leap seconds" I in fact found being used in that context. One of course also could define it to always mean the UTC leap seconds, I agree. Either way, it needs to be said :).  (UTC doesn't have a "copyright" on the terminus "leap seconds", in contrast to the terminus "UTC time", or maybe "UTC leap seconds").

I did not understand "should be amended with the proper definition of the time bases in use. I don't think that this warrants changes to the actual definitions.". This reads like yes and no. I e.g. would find it appropriate to extend the current by a GNSS_TIME_STANDARD_UNIX = 4. Or I guess, to rename to GNSS_TIME_STANDARD_UNIX = 2, and GNSS_TIME_STANDARD_UTC = 4. One of my points was that if it's not UTC it should not be called UTC. I also did not understand "The GPS timestamp is implicitly defined likewise". Is this implying that the formula I gave is not what it is supposed to be, but that it somehow is dated back to Unix epoche?

as regards PR, as mentioned, I'm a git illiterate. As regards a "set of strict definitions" I currently can't accomplish this. I mean, it is sooo much easier to spot an inconsistency, than to produce a definition in which other can't find any inconsistencies ... after all what I did was to scan the web, which doesn't make me an expert. I currently don't even seem to understand your "GPS timestamp", see above ... it's me who needs to get a definition ... suggestion: give me some formula or otherwise mathematically clear description, and I will give it a try and produce a set of definitions, which we together will then refine until perfection ... ?  ... if we could agree on the GNSS_TIME_STANDARD_XXX options ...


On yet another regard, is there a way to obtain via your great GUI tool some statistics on transmission errors and such? I looked in the Zubax docs, which I took as a guide for my slcan adapter implementation, and the mentioned SLCAN notifications don't give that. Is there maybe a set of notifications or functionality which I have not yet discovered which would yield that info?

I ask because I find that my AUAV-X2 ardupilot uavcan node is quite "unstable", especially when I have the gps messages enabled; it tends to take long to appear, and to suddenly disappear and reappear during operation. From the fact that my pair of uc4h-node and uc4h-sclan-adapter+GUITool get along together very well, I would think that my stuff is working correctly, and thus somehow suspect the issue on the ardupilot side. But considering what a hack my stuff is this is obviously a long stretch ... so info on potential faulty transmissions from my node(s) would be good.

Thx
Olli

[OlliW]

and yet another thing, sorry for being disorganized

I ask you, here, since it's PX4 stack related. Because of the mentioned instabilities with AP3.6-dev, I thought I should try a - so I presume - more stable, well established version, and thus flashed AP3.4.6.

The behavior is very reproducible, but "wired".  After a while, 10 secs or so, the system kind of freezes, I can't connect to MP, if I had been connected to MP before, no data comes in, the HUD is frozen, etc.. In the GUI Tool I see, after a reset of the AUAV-X2, the pixhawk node appearing at ID 1, and see its NodeStatus, some GetNodeInfo exchanges, some GlobalTimeSyn messages, and than esc messages at high rate. However, after this 10 secs or so these stop, and things freeze. A similar thing happens then I have the board disconnected from the CAN wires, and connect at a later point in time.

To me it looks as if the firmware is expecting to get some feedback or message, which I'm not delivering, and eventually decides to completely shut down.

So, question: What conditions do I have to provide for this to work?

(I must admit I would feel safer to fly with AP3.4.6 or AP3.5-rc5)

[Pavel Kirienko]

Hi Olli,

I just submitted a pull request with my proposed changes to the documentation. Let us continue this discussion here: https://github.com/UAVCAN/dsdl/pull/22.

> On yet another regard, is there a way to obtain via your great GUI tool some statistics on transmission errors and such?

Double click on the node, and the node window will open. Click "Get Transport Stats", and the GUI Tool will request the transport statistics from the selected node. This, obviously, will not work if the node does not implement the service uavcan.protocol.GetTransportStats.

If you were referring to the transport statistics of the local node belonging to the GUI Tool, you can do that only with the Zubax Babel at the moment. Click Tools --> CAN Adapter Control Panel, and you will see the corresponding option on the tab "Adapter State".

> I ask you, here, since it's PX4 stack related. Because of the mentioned instabilities with AP3.6-dev, I thought I should try a - so I presume - more stable, well established version, and thus flashed AP3.4.6.

My knowledge of the APM's UAVCAN stack is very limited. You should bring this question up on the APM forums instead and ping Eugene Shamaev, who is at the moment the best expert in this area.

> To me it looks as if the firmware is expecting to get some feedback or message, which I'm not delivering, and eventually decides to completely shut down.

No, your nodes are fine. The only thing required of them is that they publish NodeStatus periodically, which they seem to be doing well, so the fault is within APM. You really should be switching to the new CAN stack made by Eugene, it is much better, even though it might be not yet perfectly stable.

Thanks,

Pavel.

--
You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/4de68215-c854-4aa4-9bf1-b19d853b00dc%40googlegroups.com.

[OlliW]

Hey Pavel,
thx again for this detailed and answering answer.

error stats: ah... I haven't looked into the cli stuff so far, my fault ... yes, this was what I was looking for

cli: I like the additions very much. I'm not sure I see why it did had to come as a cli, the functionality of the available commands are not that "rich", but I guess you have a longer term vision in mind here, that's great.
I do see however one (IMHO serious) issue: According to the docs "A CLI command is a sequence of printable ASCII characters terminated with \r\n ". I read this to mean "any arbitrary" sequence of said characters terminated with \r\n. But that's then impossible to parse without a timeout or other clumsy mechanism to distinguish between the \r end of SLCAN commands and \r\n end of the cli commands (my argument hinges on the fact that this statement "The fact that CLI commands and their responses are terminated with \r\n rather than plain \r can be used to distinguish SLCAN data from CLI data in real time." is not working in practice, at an \r you need to wait for the next character to see if you can terminate, which however might come very late, or never). I thus looked into the babel code to see how you possibly could have resolved that penalty, and - if I got it right - you do that by ignoring the \n and also take \r as termination of a cli command, and then parse for the START of the message! So, (1) the statement in the docs is incorrect, (2) it's IMHO a most unfortunate specification = requires an "ugly" parser (I know a bit about parsers/languages/compilers).

Sad you didn't decide to choose just \n as termination for the cli commands (the terminals I know all allow to adjust their display for \r, \n, \r\n).

At this point I should admit that I haven't found the definitions of these LIKELY, UNLIKELY macros; in case they contain some secret sauce such as a lookahead mechanism to see if a \n might be the next character, my statements are nevertheless valid (and I would conclude that your parser can't be efficient).

As a side effect - which is of course not your concern - my parser, which parses for \r, isn't easily extended ... I guess I will go for just ignoring incoming \n characters ...

AP: thx, I will do as you advice.

Cheers, Olli

[Pavel Kirienko]

Hi Olli,

You are absolutely right in pointing out that the approach based on \r\n requires a very complicated parser, but it was designed this way intentionally. My objective was to enable co-existence of the "standard" CLI (like VT100) and SLCAN on the same interface, therefore I was not at liberty to choose an arbitrary line termination sequence. Your statement that a delay will be required to distinguish CLI from SLCAN is not correct, as it is possible to design a state machine that will be separating the two at zero time penalty. Such a parser is implemented in PyUAVCAN.

> I thus looked into the babel code to see how you possibly could have resolved that penalty, and - if I got it right - you do that by ignoring the \n and also take \r as termination of a cli command, and then parse for the START of the message!

It doesn't matter how exactly the specification is implemented; what matters is that the observable behavior of the protocol is indistinguishable from what is described in the specification. Yes, Babel parses the packets speculatively in order to minimize latency and maximize the throughput, but this implementation detail does not concern the user, because it provides a specification-compliant interface.

> At this point I should admit that I haven't found the definitions of these LIKELY, UNLIKELY macros

The macros LIKELY and UNLIKELY are simple static branch hinting constructs, they are used to optimize the performance of the ROM prefetcher (Babel is a very performance sensitive device due to the latency and throughput concerns). They expand to __builtin_expect().

Pavel.

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/7179ed3c-05a6-4404-ba53-4be6dfe738bc%40googlegroups.com.

[OlliW]

Hey Pavel,

I understand your desired to get a cli integrated and that this has enforces restrictions. I'm not familiar with "standard" CLIs like VT100, just with standard uart terminals such as HTerm ... so, these "standard CLIs" you are referring to do not allow a configuration such as to expand incoming/outgoing \r into \r\n ?   (I think I'm glad to not use this sort of tools hihi).

I do not agree with that \r \r\n thing.
At the point in time when a \r has arrived, you cannot know when a next character arrives and what character this would be. It's such a simple statement that it hardly can be denied.
One can do workarounds: One could wait = time delay. One could have a time out mechanism = time delay. One could analyze the received command string = that's what you do, no time penalty, but it has ambiguity and requires definitions to resolve them.
Let me wrap it this way and ask: Please provide me the parser which separates the SLCAN command T0123456780102030405060708\r from  the cli command T0123456780102030405060708\r\n with zero time penalty in all possible situations. The specification allows that.
I think I can safely write: Q.E.D. :)

I fully agree that it doesn't matter how things are implemented, but Babel will fail when I would send the just mentioned cli command; by observing its behavior I could tell that it is not specification compliant.

If changing the specification from \r\n to \n is not a viable option (which I guess you will conclude is not), maybe one could add to the specification a simple requirement, such as that the first character of a cli command must not coincide with a SLCAN character, or first two characters (which I guess would in principle allow even faster state machines, wouldn't this be LL(1) or LL(2) grammars).  (and drop the conflicting \r\n statement)

It used to be a hobby of mine to do this parser/compiler/language/grammar stuff when I was in my twenties. So, I very much liked this discussion. Even though it's of little practical relevance. :)

Anyway, I think I understand now what I need to do to integrate the cli, and to get to my error stats. Thx a lot, sir.

Cheers, Olli

[OlliW]

btw: I think I've at least "sorted" out the AP3.4.6 UAVCAN/PX4 issue ... it seems it doesn't like too inconsistent magnetometers, and freezes when without alert ... whatever, since I have them in better consistency, it runs now for quite a while on my bench ...

[OlliW]

sad one can't edit here

edit: no ... this must have been coincidental .. the situation is weird, at least I can't figure it out it seems ... I guess I have given up with the project

:)

[Pavel Kirienko]

Hi Olli,

Concerning APM, I have very little control over what is happening over there, so the best advice I can give is to relay your reports to their Discourse (which you already did it seems).

Regarding the parser issue: you are very right by saying that the approach employed by Babel and PyUAVCAN breaks apart in a general case, and this is, indeed, a specification issue. While working on Babel, I failed to validate the specification independently from the particular implementation at hand; now I am going to amend it. Thank you for pointing that out!

Pavel.

On Sat, May 6, 2017 at 1:14 PM, OlliW <oll...@gmail.com> wrote:

btw: I think I've at least "sorted" out the AP3.4.6 UAVCAN/PX4 issue ... it seems it doesn't like too inconsistent magnetometers, and freezes when without alert ... whatever, since I have them in better consistency, it runs now for quite a while on my bench ...

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/70bb3a05-3ce5-4586-ad10-3d665d0bef7e%40googlegroups.com.

[OlliW]

Hey Pavel,

the docu change makes it clear. I think you also need to remove/change the sentence further below "The fact that CLI commands and their responses are terminated with \r\n rather than plain \r can be used to distinguish SLCAN data from CLI data in real time". It is the condition that a cli command must not match a slcan command which allows that.

Just as a side comment, not important. I found the statement "Every CLI command returns a response, which may be empty. Response begins with the exact copy of the received command..." somewhat confusing ... how can the response be empty if it must start with the exact copy.

As regards APM, well, I certainly don't want to put blame on them ... I mean, for them it works nicely as it seems ... it's just that I can't figure it out what's going on on my desk
(I get a sense that in parts it might be the SN65HVD230, or 3.3V CAN transceivers ... today I got the TJA1051T/3 for my uc4h node pcb, so I quickly soldered it, and it gives so much cleaner CAN signals on the oszi, and it gives signals which look like I would have thought CAN signals look like, unlike with the SN65HVD230 chips. I've quickly checked, and pixracer also uses them. Anyway.)

Cheers, Olli

[Pavel Kirienko]

Thanks Olli, the latest concerns were addressed.

Most Zubax devices are based on SN65HVD230, and this chip seems to work flawlessly. So the problem must be elsewhere.

Pavel.

--

You received this message because you are subscribed to the Google Groups "UAVCAN" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uavcan+unsubscribe@googlegroups.com.
To post to this group, send email to uav...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/uavcan/d1766ac1-1743-4129-89ff-2a43b542c8af%40googlegroups.com.

[OlliW]

yeah ... I'm likely clinging to straws ... :)

in any case, the TJA is a really nice device, IMHO, I guess I'm going to prefer it much over the SN ... and it is available in a small package

Thx, Olli