Data format on CANbus???
20 views
Skip to first unread message
Crashmatt
Sep 25, 2010, 2:56:25 AM9/25/10
to RC servo interface
Now there is some time out from hardware development we are back to
the software system side.
Our basic goal is to mirror the variables in the autopilot with the
variables in the interface so the data can be sent to telemetry or
another processor.
There are three ways to do this:
1. Reference by memory location (Suggestion from Paul G)
2. Reference by identifier (My suggestion based on other protocols)
3. Someone (Jeremy?) fixes MAVlink over CANbus
There are pros and cons to each
Ref by memory location:
Super simple. Lowest resource overhead.
Requires reorganisation of memory variables into structures and
fixed locations.
Both end of the link need to know absolute memory location.
Ref by identifier:
Hardware independant. An easy plug-in to existing code.
More complex to build. Requires mainenance of identifier list.
Transmitted structures are limited to 16bytes maximum. Large
arrays or ok.
MAVLink over CANbus
We relax and use a well known system.
We are likely to be locked into MAVLink.
I have already built (untested) something like the ref by identifier
code. It was intended for telemetry communication but could be
modified for CANbus.
Communication on the telemetry side is no problem. We pick up the
autopilot telemetry code and put it in the interface.
I will continue to re-write my code until somebody does or suggests
something to convice me otherwise.
Many thanks to Paul G on some long conversations on telemetry
protocol. I am still not conviced that my suggestion is better than
his.
We might have problems read/writing data for communication that is
being modified by the autopilot, resulting in junk data. We need a
low resource fix for this.
As always, any opinions on these topics are welcome.
the software system side.
Our basic goal is to mirror the variables in the autopilot with the
variables in the interface so the data can be sent to telemetry or
another processor.
There are three ways to do this:
1. Reference by memory location (Suggestion from Paul G)
2. Reference by identifier (My suggestion based on other protocols)
3. Someone (Jeremy?) fixes MAVlink over CANbus
There are pros and cons to each
Ref by memory location:
Super simple. Lowest resource overhead.
Requires reorganisation of memory variables into structures and
fixed locations.
Both end of the link need to know absolute memory location.
Ref by identifier:
Hardware independant. An easy plug-in to existing code.
More complex to build. Requires mainenance of identifier list.
Transmitted structures are limited to 16bytes maximum. Large
arrays or ok.
MAVLink over CANbus
We relax and use a well known system.
We are likely to be locked into MAVLink.
I have already built (untested) something like the ref by identifier
code. It was intended for telemetry communication but could be
modified for CANbus.
Communication on the telemetry side is no problem. We pick up the
autopilot telemetry code and put it in the interface.
I will continue to re-write my code until somebody does or suggests
something to convice me otherwise.
Many thanks to Paul G on some long conversations on telemetry
protocol. I am still not conviced that my suggestion is better than
his.
We might have problems read/writing data for communication that is
being modified by the autopilot, resulting in junk data. We need a
low resource fix for this.
As always, any opinions on these topics are welcome.
Paul G.
Sep 25, 2010, 9:49:44 AM9/25/10
to RC servo interface
The ref by memory location does not require any reorganization of
existing variables, setting the optional address attributes on them
will mean they won't all move when something is added - instead the
compiler will place the variables into the "holes" left between the
ones that do have their addresses set. For a given (and thereby
unchanging) set of messages if the compiler places the referenced
variables in the same memory locations the software could have radical
changes and not affect the GCS - this is the only scenario were this
is true. You also gain good immunity from version skew in this case,
because your GCS does not need to be updated every time you change the
IMU AND visa versa. You thought fixing variable locations was not
likely to happen, how much less likely is it to have all the variable
names changed to what MAVLink wants, ignoring the fact that the
variables are not even the same type in the first place.
I had a look at MAVLink - I did not see anything of great import - and
its just about all float based, meaning you'd have to translate just
about every request, something that's not likely to fit in the
existing 2k ram on a 4011 - but something the PC can do very easily -
it has the memory and cpu cycles to burn.
According to their website your not stuck with MAVLink to use
QGroundControl (the actual GCS) - that's the way I would go. You could
even keep all the message numbers the same, kill some of the silly
overhead and keep normal byte orders in the comm side messages (little
endian vs big endian). Or once it arrives in the PC do format
conversions ie ints -> floats as required, then do endian conversion
then emit the bytes to existing MAVLink were it can do what it needs
to do (basically undo everything you just did). You will end up
writing a bunch of code that's busy work but spread sheets are real
good at that :)
Bottom line IMO don't bother emulating the byte stream at the link
level, you gain nothing by it, emulate it at the function level on the
PC were its easy. Memory image is the best way bar none on CAN, use
the extended ID bits as address's on both sides of the link, and const
tables to pick the ones you need to read or just cycle through the
entire mem array. 2K of memory is 16k bits, on a 1 megabit CAN link
even after the overhead your reading the entire thing at 40hz+
ignoring the fact that about 25% of it is stack space and does not
need to be read at all. You still need the const tables to put the
content in the message or you have to write a bunch of custom
functions - the tables will take less space and be easier to maintain
and still execute well before the the header has even transmitted.
The one really negative thing I did see is the message numbering with
push data - although the consequence is just the error counter will go
up. Its truly amassing to me people treat RF links like wired links -
RF is half duplex, and every time you switch directions you incur huge
overhead (time wise). Push data for at least 50% of the bandwidth will
produce nice increases in apparent responsiveness and total throughput
- but you need to let it idle some or you'll never receive any
messages.
Paul
existing variables, setting the optional address attributes on them
will mean they won't all move when something is added - instead the
compiler will place the variables into the "holes" left between the
ones that do have their addresses set. For a given (and thereby
unchanging) set of messages if the compiler places the referenced
variables in the same memory locations the software could have radical
changes and not affect the GCS - this is the only scenario were this
is true. You also gain good immunity from version skew in this case,
because your GCS does not need to be updated every time you change the
IMU AND visa versa. You thought fixing variable locations was not
likely to happen, how much less likely is it to have all the variable
names changed to what MAVLink wants, ignoring the fact that the
variables are not even the same type in the first place.
I had a look at MAVLink - I did not see anything of great import - and
its just about all float based, meaning you'd have to translate just
about every request, something that's not likely to fit in the
existing 2k ram on a 4011 - but something the PC can do very easily -
it has the memory and cpu cycles to burn.
According to their website your not stuck with MAVLink to use
QGroundControl (the actual GCS) - that's the way I would go. You could
even keep all the message numbers the same, kill some of the silly
overhead and keep normal byte orders in the comm side messages (little
endian vs big endian). Or once it arrives in the PC do format
conversions ie ints -> floats as required, then do endian conversion
then emit the bytes to existing MAVLink were it can do what it needs
to do (basically undo everything you just did). You will end up
writing a bunch of code that's busy work but spread sheets are real
good at that :)
Bottom line IMO don't bother emulating the byte stream at the link
level, you gain nothing by it, emulate it at the function level on the
PC were its easy. Memory image is the best way bar none on CAN, use
the extended ID bits as address's on both sides of the link, and const
tables to pick the ones you need to read or just cycle through the
entire mem array. 2K of memory is 16k bits, on a 1 megabit CAN link
even after the overhead your reading the entire thing at 40hz+
ignoring the fact that about 25% of it is stack space and does not
need to be read at all. You still need the const tables to put the
content in the message or you have to write a bunch of custom
functions - the tables will take less space and be easier to maintain
and still execute well before the the header has even transmitted.
The one really negative thing I did see is the message numbering with
push data - although the consequence is just the error counter will go
up. Its truly amassing to me people treat RF links like wired links -
RF is half duplex, and every time you switch directions you incur huge
overhead (time wise). Push data for at least 50% of the bandwidth will
produce nice increases in apparent responsiveness and total throughput
- but you need to let it idle some or you'll never receive any
messages.
Paul
Crashmatt
Sep 26, 2010, 6:49:11 PM9/26/10
to RC servo interface
Thanks to Paul for his communication. As usuall his posts are full of
good ideas but they take considerable time to digest.
1. I agree that changing structures or re-organising the code is not
necessary for address identifiers.
2. In my opinion, we should not attempt to branch off a version of
matrixpilot code for this project. It would not benefit the standard
users and it becomes difficult to maintain. For this reason, any
changes to the main code base would need full understanding and
support from the whole community.
3. A dark side of reference by identifier is bad transmission
efficiency.
This is a long topic so I will make a document for it. It will show us
how much bandwidth to expect.
3. A key aim is to have a system that can easily share the variable
locations between the communication ends. An ideal is not to have to
manually maintain two sets of code.
The memory reference scheme effectively creates a database of memory
locations within the code. The output from this databsae is the .map
file which could be useful.
The identifier scheme shares a common database from which variable
address lookup tables are built. Different tables can be built for
different points in the system.
4. We need to take some care over splitting up variables as they are
pushed. A minor issue for address based push for which I feel that
there is a simple solution
5. I would like to avoid thinking of this as a groundstation telemetry
communication. It is perhaps overcomplicating what needs to be done.
Once the data is transfered to/from the autopilot processor then the
choice after that is open.
One of the open choices is Petes telemetry format which works nicely
with the four letter acronymed data processor he wrote in python.
6. As long as this data transport thing is kept modular and open, it
does not matter which one gets used. We only need one working one.
I would like to see some examples of projects using these different
ways of communicating. Can we find anything in the open source
community and get feedback from the users.
To be conviced of memory reference it would be good to see some
demonstation code. Something that shows what needs to go where and
how it would be maintained.
Best regards
Matt
good ideas but they take considerable time to digest.
1. I agree that changing structures or re-organising the code is not
necessary for address identifiers.
2. In my opinion, we should not attempt to branch off a version of
matrixpilot code for this project. It would not benefit the standard
users and it becomes difficult to maintain. For this reason, any
changes to the main code base would need full understanding and
support from the whole community.
3. A dark side of reference by identifier is bad transmission
efficiency.
This is a long topic so I will make a document for it. It will show us
how much bandwidth to expect.
3. A key aim is to have a system that can easily share the variable
locations between the communication ends. An ideal is not to have to
manually maintain two sets of code.
The memory reference scheme effectively creates a database of memory
locations within the code. The output from this databsae is the .map
file which could be useful.
The identifier scheme shares a common database from which variable
address lookup tables are built. Different tables can be built for
different points in the system.
4. We need to take some care over splitting up variables as they are
pushed. A minor issue for address based push for which I feel that
there is a simple solution
5. I would like to avoid thinking of this as a groundstation telemetry
communication. It is perhaps overcomplicating what needs to be done.
Once the data is transfered to/from the autopilot processor then the
choice after that is open.
One of the open choices is Petes telemetry format which works nicely
with the four letter acronymed data processor he wrote in python.
6. As long as this data transport thing is kept modular and open, it
does not matter which one gets used. We only need one working one.
I would like to see some examples of projects using these different
ways of communicating. Can we find anything in the open source
community and get feedback from the users.
To be conviced of memory reference it would be good to see some
demonstation code. Something that shows what needs to go where and
how it would be maintained.
Best regards
Matt
J. L.
Sep 26, 2010, 7:08:55 PM9/26/10
to rc-servo-...@googlegroups.com
I have been digesting it as well the last few days and trying to come
up with a few ideas, though for me the 2 ways I will be using it will
be totally different as for the overo and most of my ideas previously
were about that.
I agree avoiding branching off the main line is the ideal way I think
for sure. I have been swamped with some other projects but will have
more time tomorrow and teus to give a more proper answer with details
of how I was thinking for the overo use and maybe could be something
similar done with matrix. But I will be honest I have never delt with
the can bus programming before. But hey pretty much all of this stuff
I have not done before starting with UAV's and I love to learn how to
code something new.
I have gotten away from trying to make and use mavlink in my setup as
just was not what I wanted
Crashmatt
Sep 27, 2010, 6:55:59 AM9/27/10
to RC servo interface
Jeremy, I thought you were using MAVlink. I don't think any of us are
hooked on MAVLink so it is no loss if we use something else.
Everyone,
I have a suggestion for a solution to keep all sides happy:
We define a few bits in the CANbus message identifier to indicate
which protocol is being transmitted. 3 bits should be enough.
There is already one protocol running to transfer servo/rx data. I am
not planning to change this mechanism. It should be completely
independant of other data transfers.
Standard CANbus devices have postboxes that receive messages. There
are two receiver postboxes. One will be devoted to critical control
data.
The CANbus message identifier bits act as a priority. CANbus has so
called dominant and recessive bits which is part of the physical
layer. An identifier with most dominant bits wins the priority
battle.
The servo data is considered the highest priority critical control
data. It has the highest priority identifier.
After the servo identifier we can define and use whatever we like. If
we wish to use a mix of Pauls, mine, Bobs and Freds protocols, thats
no problem.
If we now define the interface functions for transmitting and
receiving CANbus data, changing protocol code should be easy.
Regards Matt
On Sep 27, 1:08 am, "J. L. " <vwyodap...@gmail.com> wrote:
hooked on MAVLink so it is no loss if we use something else.
Everyone,
I have a suggestion for a solution to keep all sides happy:
We define a few bits in the CANbus message identifier to indicate
which protocol is being transmitted. 3 bits should be enough.
There is already one protocol running to transfer servo/rx data. I am
not planning to change this mechanism. It should be completely
independant of other data transfers.
Standard CANbus devices have postboxes that receive messages. There
are two receiver postboxes. One will be devoted to critical control
data.
The CANbus message identifier bits act as a priority. CANbus has so
called dominant and recessive bits which is part of the physical
layer. An identifier with most dominant bits wins the priority
battle.
The servo data is considered the highest priority critical control
data. It has the highest priority identifier.
After the servo identifier we can define and use whatever we like. If
we wish to use a mix of Pauls, mine, Bobs and Freds protocols, thats
no problem.
If we now define the interface functions for transmitting and
receiving CANbus data, changing protocol code should be easy.
Regards Matt
On Sep 27, 1:08 am, "J. L. " <vwyodap...@gmail.com> wrote:
J. L.
Sep 27, 2010, 12:02:14 PM9/27/10
to rc-servo-...@googlegroups.com
I was using it yes but found it and qgroundcontrol lacking for my
needs.. I still have my modified version if you wanted to see it
needs.. I still have my modified version if you wanted to see it
Peter Hollands
Sep 27, 2010, 12:06:25 PM9/27/10
to rc-servo-...@googlegroups.com
Jeremy,
I'm very interested in using mavlink. I would be interested in understanding something about your own requirements,
and why Mavlink was not appropriate (even with modifications).
It would seem that it provided two signifiant features:
1) No real need to parse data combined with easy synchronization of changes to the format.
2) A binary protocol which was efficient and incorporated sum checks.
Best wishes, Pete
I'm very interested in using mavlink. I would be interested in understanding something about your own requirements,
and why Mavlink was not appropriate (even with modifications).
It would seem that it provided two signifiant features:
1) No real need to parse data combined with easy synchronization of changes to the format.
2) A binary protocol which was efficient and incorporated sum checks.
Best wishes, Pete
Crashmatt
Sep 27, 2010, 1:52:21 PM9/27/10
to RC servo interface
Here is a long investigation into the bandwidth we can get from
CANbus.
http://groups.google.com/group/rc-servo-interface/web/Servo%20Interface%20-%20Rough%20guide%20to%20data%20bandwidth.pdf
We will get around 200bytes of transfer at 50Hz. This is plenty of
bandwidth for any binary based protocol we wish to run on it.
On Sep 27, 6:06 pm, Peter Hollands <peter.holla...@gmail.com> wrote:
> Jeremy,
>
> I'm very interested in using mavlink. I would be interested in understanding
> something about your own requirements,
> and why Mavlink was not appropriate (even with modifications).
>
> It would seem that it provided two signifiant features:
>
> 1) No real need to parse data combined with easy synchronization of changes
> to the format.
> 2) A binary protocol which was efficient and incorporated sum checks.
>
> Best wishes, Pete
>
> ...
>
> read more »
CANbus.
http://groups.google.com/group/rc-servo-interface/web/Servo%20Interface%20-%20Rough%20guide%20to%20data%20bandwidth.pdf
We will get around 200bytes of transfer at 50Hz. This is plenty of
bandwidth for any binary based protocol we wish to run on it.
On Sep 27, 6:06 pm, Peter Hollands <peter.holla...@gmail.com> wrote:
> Jeremy,
>
> I'm very interested in using mavlink. I would be interested in understanding
> something about your own requirements,
> and why Mavlink was not appropriate (even with modifications).
>
> It would seem that it provided two signifiant features:
>
> 1) No real need to parse data combined with easy synchronization of changes
> to the format.
> 2) A binary protocol which was efficient and incorporated sum checks.
>
> Best wishes, Pete
>
> On Mon, Sep 27, 2010 at 5:02 PM, J. L. <vwyodap...@gmail.com> wrote:
> > I was using it yes but found it and qgroundcontrol lacking for my
> > needs.. I still have my modified version if you wanted to see it
>
> > On Mon, Sep 27, 2010 at 3:55 AM, Crashmatt <crash__m...@hotmail.com>
> > I was using it yes but found it and qgroundcontrol lacking for my
> > needs.. I still have my modified version if you wanted to see it
>
>
> read more »
J. L.
Sep 28, 2010, 1:57:11 PM9/28/10
to rc-servo-...@googlegroups.com
On Mon, Sep 27, 2010 at 9:06 AM, Peter Hollands
<peter.h...@gmail.com> wrote:
> Jeremy,
>
> I'm very interested in using mavlink. I would be interested in understanding
> something about your own requirements,
> and why Mavlink was not appropriate (even with modifications).
<peter.h...@gmail.com> wrote:
> Jeremy,
>
> I'm very interested in using mavlink. I would be interested in understanding
> something about your own requirements,
> and why Mavlink was not appropriate (even with modifications).
My project is not like what most people are doing for their
autopilots, my drones are security testing drones that are actively
doing activities to audit networks, wifi, bluetooth, cell, and things
like that. Also I have 2 streaming videos and 2 more that are just
logged on board. I found that with everything I am trying to stream
back to my ground station was not best suited with what they have
done. I agree with the basics that most people need its a great setup,
but I am in the minority with 98% of needs for my project. I started
rewriting my own protocol that goes over the N band for wifi that can
fit everything I need in their. This is for my main telemetry info, I
also have a back up that sends just the most basic info about the AP
as well in case I get out of range with the wifi connection or
something similar.
getting checksums is not that hard in any protocol that you use at
least its that way for me. Hope that gives a better explanation for
you Pete.
I try not to talk much about my project as its caused me a ton of
grief on diydrones, private message threats that I am ruining the name
of AP's for hobby use, and endless BS like that, finally got to the
point I just dont ask questions about anything none AP related.
J. L.
Sep 28, 2010, 2:06:22 PM9/28/10
to rc-servo-...@googlegroups.com
OFF TOPIC: sorry if this is bad etiquette.
Also I was not happy with the qgrondcontrol software and found myself
having to tweak to much of it to get what I wanted displayed. I have a
great idea for a GCS that could become a universal piece of software
for all AP's but if I can not find anyone who would want to work on
the same thing with me I doubt I could code it to what my idea is
mostly because of time and if its just for me then there is no real
need to make things pretty and worry about others needs.
But my idea for a GCS is this, its something similar to what open
pilot is trying to do with their GCS, but what I want to do is make
GCS that you can load specific info in about your AP, say with the UDB
the people that use UDB could come up with everything they need for
their GCS as well as debugging and programming and we could
incorporate that into a UDB plug in and the same would be for any
other AP out there the project can come up with their own plugins to
give the same for theirs. Basically it would be a solid backbone for
all AP's but its customized with the plugins. Also by giving the
abilities to add in plug ins someone who needs something specific for
their project can add one for their personal needs and still have all
the info for their AP. Hope this all made sense was trying to give a
quick run down on the idea. If this sounds good let me know and I can
post a full break down of my thoughts in a post on the UDB
messageboard as its probably more suited to talk there.
Also sorry Matt if this was rude to do this off topic post here.
Crashmatt
Sep 28, 2010, 5:15:12 PM9/28/10
to RC servo interface
Jeremy,
I now have bandwidth envy. My little XBee modules are looking pretty
weak.
Now trying to re-reorganise the CANbus messages. The dsPIC CAN
registers are very badly organised. The identifier is spread across
three registers in different patterns for transmit and receive. Much
confusion.
Matt
On Sep 28, 7:57 pm, "J. L. " <vwyodap...@gmail.com> wrote:
> On Mon, Sep 27, 2010 at 9:06 AM, Peter Hollands
>
> ...
>
> read more »
I now have bandwidth envy. My little XBee modules are looking pretty
weak.
Now trying to re-reorganise the CANbus messages. The dsPIC CAN
registers are very badly organised. The identifier is spread across
three registers in different patterns for transmit and receive. Much
confusion.
Matt
On Sep 28, 7:57 pm, "J. L. " <vwyodap...@gmail.com> wrote:
> On Mon, Sep 27, 2010 at 9:06 AM, Peter Hollands
>
> > On Mon, Sep 27, 2010 at 5:02 PM, J. L. <vwyodap...@gmail.com> wrote:
>
> >> I was using it yes but found it and qgroundcontrol lacking for my
> >> needs.. I still have my modified version if you wanted to see it
>
> >> On Mon, Sep 27, 2010 at 3:55 AM, Crashmatt <crash__m...@hotmail.com>
>
> >> I was using it yes but found it and qgroundcontrol lacking for my
> >> needs.. I still have my modified version if you wanted to see it
>
>
> read more »
Peter Hollands
Sep 29, 2010, 9:25:00 AM9/29/10
to rc-servo-...@googlegroups.com
Jeremy,
Thanks for taking the time to explain why Qgroundcontrol and mavlink are not for you.
I'm trying to avoid writing a ground control station (GCS) for as long as possible !
I shall take a longer look at mavlink over the next couple of weeks. We need some type of binary protocol to send larger amounts of telemetry data at least 40 times / second from the UDB. We've done well without it, but really, we should be seeing more of our foundation data than we do.
Best wishes, Pete
Thanks for taking the time to explain why Qgroundcontrol and mavlink are not for you.
I'm trying to avoid writing a ground control station (GCS) for as long as possible !
I shall take a longer look at mavlink over the next couple of weeks. We need some type of binary protocol to send larger amounts of telemetry data at least 40 times / second from the UDB. We've done well without it, but really, we should be seeing more of our foundation data than we do.
Best wishes, Pete
Crashmatt
Sep 29, 2010, 12:34:37 PM9/29/10
to RC servo interface
Pete,
Regarding your thought on MAVLink saving us from writing parsers.
Here are my thoughts (perhaps incorrect) on it:
Regardless of what language is spoken over the telemetry, something
somewhere needs to translate between autopilot and groundstation
variables. The only way to get around this is if the groundstation
and autopilot have common variable types in the same units. We can
not do this with the PIC integer maths.
The translation between variables could be done in many different
ways:
Coded piece by piece parsing
database lookup tables
xml definitions
Sticking with the database and reference by identifiers idea:
The database could have entries for groundstation variables that
correspond to the autopilot variables.
Give each identifier/variable a scaling and offset factor to turn it
into SI units.
Give each identifier/variable a scaling and offset factor to turn it
from SI units into groundstation units.
Give each identifier/variable a pointer to a function that will
translate between autopilot and groundstation variable types.
Give each identifier/variable a reference to the groundstation
identifier reference (if they are not the same)
Thats as close to automated parsing as I can think of. Maybe xml
would be a better way of defining the database.
When someone changes the groundstation, the translation must change.
This is a good reason not to do hard coded parsing.
The database adds more layers of compexity. This is a good reason to
do hard coded parsing.
Am Ì making this too complicated?
Regards Matt
On Sep 29, 3:25 pm, Peter Hollands <peter.holla...@gmail.com> wrote:
> Jeremy,
>
> Thanks for taking the time to explain why Qgroundcontrol and mavlink are not
> for you.
>
> I'm trying to avoid writing a ground control station (GCS) for as long as
> possible !
>
> I shall take a longer look at mavlink over the next couple of weeks. We need
> some type of binary protocol to send larger amounts of telemetry data at
> least 40 times / second from the UDB. We've done well without it, but
> really, we should be seeing more of our foundation data than we do.
>
> Best wishes, Pete
>
> On Tue, Sep 28, 2010 at 7:06 PM, J. L. <vwyodap...@gmail.com> wrote:
> ...
>
> read more »
Regarding your thought on MAVLink saving us from writing parsers.
Here are my thoughts (perhaps incorrect) on it:
Regardless of what language is spoken over the telemetry, something
somewhere needs to translate between autopilot and groundstation
variables. The only way to get around this is if the groundstation
and autopilot have common variable types in the same units. We can
not do this with the PIC integer maths.
The translation between variables could be done in many different
ways:
Coded piece by piece parsing
database lookup tables
xml definitions
Sticking with the database and reference by identifiers idea:
The database could have entries for groundstation variables that
correspond to the autopilot variables.
Give each identifier/variable a scaling and offset factor to turn it
into SI units.
Give each identifier/variable a scaling and offset factor to turn it
from SI units into groundstation units.
Give each identifier/variable a pointer to a function that will
translate between autopilot and groundstation variable types.
Give each identifier/variable a reference to the groundstation
identifier reference (if they are not the same)
Thats as close to automated parsing as I can think of. Maybe xml
would be a better way of defining the database.
When someone changes the groundstation, the translation must change.
This is a good reason not to do hard coded parsing.
The database adds more layers of compexity. This is a good reason to
do hard coded parsing.
Am Ì making this too complicated?
Regards Matt
On Sep 29, 3:25 pm, Peter Hollands <peter.holla...@gmail.com> wrote:
> Jeremy,
>
> Thanks for taking the time to explain why Qgroundcontrol and mavlink are not
> for you.
>
> I'm trying to avoid writing a ground control station (GCS) for as long as
> possible !
>
> I shall take a longer look at mavlink over the next couple of weeks. We need
> some type of binary protocol to send larger amounts of telemetry data at
> least 40 times / second from the UDB. We've done well without it, but
> really, we should be seeing more of our foundation data than we do.
>
> Best wishes, Pete
>
> > >> On Mon, Sep 27, 2010 at 5:02 PM, J. L. <vwyodap...@gmail.com> wrote:
>
> > >>> I was using it yes but found it and qgroundcontrol lacking for my
> > >>> needs.. I still have my modified version if you wanted to see it
>
> > >>> On Mon, Sep 27, 2010 at 3:55 AM, Crashmatt <crash__m...@hotmail.com>
>
> > >>> I was using it yes but found it and qgroundcontrol lacking for my
> > >>> needs.. I still have my modified version if you wanted to see it
>
>
> read more »
Crashmatt
Sep 30, 2010, 12:55:44 AM9/30/10
to RC servo interface
I am starting to think that some structures will be larger than the
CANbus 8 byte limit.
An example is an absolute waypoint:
Earth position to 1m is roughly 25bits each for lat and long.
Rounding up you get to 4 bytes each, 8 bytes total.
Then add a few bytes for waypoint settings.
There are three alternatives I know of:
1. Use Pauls address based identifier and be very careful not to read
or write parts of a variable.
2. Make CANbus capable of carrying larger structures
Limits the address space
Message buffers need to be implemented.
Be very careful to make sure that the buffer is all filled from the
same variable
3. Úse an addressing variable.
A variable sets the currently selected structure (waypoint)
Parts of the waypoint can be modified
64bit address space instead of ¨8bit.
Extension of simple scheme
Needs more bandwidth.
Variables do not need to be arranged on a specific byte boundary.
of which #3 is my favorite because it is the safest and is of least
concern to the main code.
It can also be implemented at a later date. We do not need to worry
about it now.
Matt
CANbus 8 byte limit.
An example is an absolute waypoint:
Earth position to 1m is roughly 25bits each for lat and long.
Rounding up you get to 4 bytes each, 8 bytes total.
Then add a few bytes for waypoint settings.
There are three alternatives I know of:
1. Use Pauls address based identifier and be very careful not to read
or write parts of a variable.
2. Make CANbus capable of carrying larger structures
Limits the address space
Message buffers need to be implemented.
Be very careful to make sure that the buffer is all filled from the
same variable
3. Úse an addressing variable.
A variable sets the currently selected structure (waypoint)
Parts of the waypoint can be modified
64bit address space instead of ¨8bit.
Extension of simple scheme
Needs more bandwidth.
Variables do not need to be arranged on a specific byte boundary.
of which #3 is my favorite because it is the safest and is of least
concern to the main code.
It can also be implemented at a later date. We do not need to worry
about it now.
Matt
UAVflightdirector
Sep 30, 2010, 2:21:57 AM9/30/10
to RC servo interface
Here is a link to a database spreadsheet linking variable to
identifiers.
https://spreadsheets.google.com/ccc?key=0ApSF4VCMDng5dDFpamx1MVlOY2JxcUxGa3FOcXQ2a2c&hl=en&authkey=CMO2z90N
This is used to generate headers and lookup tables.
This spreadsheet is still very much under construction
identifiers.
https://spreadsheets.google.com/ccc?key=0ApSF4VCMDng5dDFpamx1MVlOY2JxcUxGa3FOcXQ2a2c&hl=en&authkey=CMO2z90N
This is used to generate headers and lookup tables.
This spreadsheet is still very much under construction
Paul G.
Sep 30, 2010, 8:24:12 AM9/30/10
to RC servo interface
Please see comments in the quote.
On Sep 29, 9:55 pm, Crashmatt <crash__m...@hotmail.com> wrote:
> I am starting to think that some structures will be larger than the
> CANbus 8 byte limit.
>
> An example is an absolute waypoint:
> Earth position to 1m is roughly 25bits each for lat and long.
> Rounding up you get to 4 bytes each, 8 bytes total.
> Then add a few bytes for waypoint settings.
>
> There are three alternatives I know of:
>
> 1. Use Pauls address based identifier and be very careful not to read
> or write parts of a variable.
You may - in fact you will - read parts of a variable - its very
important not to write parts of a variable. PIC has a disable
interrupts opcode so that's covered, what's not covered is switching
from using one set of variables to another. That needs to be done at a
higher level. As an example don't update the "live" way point, update
the next one and then switch to it so the high level AP code knows to
re-calculate targets, note flag changes and all those things that are
more or less one time events on way point switch rather than it having
to execute that code ever single navigation cycle.
>
> 2. Make CANbus capable of carrying larger structures
> Limits the address space
> Message buffers need to be implemented.
> Be very careful to make sure that the buffer is all filled from the
> same variable
>
CAN is meant for low latency - that's at odds with throughput were
your interested in getting lots of information. CAN message lengths
are set and will require silicon changes that are not going to happen.
It would have been nice to get 16 byte messages or have the DLC field
as power of 2 coding but it not. A simple trick to making sure you
have a coherent copy of a complex variable is to frame it on each side
with a key value eg when updating a way point you start building a
buffer by clearing all the contents, then transfer the actual values
into the middle part and then update both frame values - of course 0
should not be considered a valid frame "key" value in that case. This
would be a minimum for any control type messages. The actual key
values could be CRC values or hard coded and normal packet
verification should still happen.
> 3. Úse an addressing variable.
> A variable sets the currently selected structure (waypoint)
> Parts of the waypoint can be modified
> 64bit address space instead of ¨8bit.
> Extension of simple scheme
> Needs more bandwidth.
> Variables do not need to be arranged on a specific byte boundary.
>
64 bit address space ? the biggest pic in existence has 128k of ram -
that's 17 bit's of address. Variables will always need to be on word
boundaries if they are larger than one byte - that's part of the PIC
architecture.
> of which #3 is my favorite because it is the safest and is of least
> concern to the main code.
> It can also be implemented at a later date. We do not need to worry
> about it now.
>
> Matt
Regarding data format conversion - never do it on the AP, its
completely pointless, do it on the PC. The variables on the AP are by
definition as precise as they can be, making them smaller you lose
precision and making them bigger at best adds zero's that take longer
to transmit. The communication routine is always going to have to know
what its getting - trying to come up with a universal number format is
like trying to fit a square peg in a round hole and works for neither
the peg or hole. This gets mitigated at the source some because we try
to use all the precision available in the source variables but at a
fundamental level velocity is not equal to say altitude (the possible
ranges are just too different). If we were using some really high end
DSP were everything was 64bit floating point values we'd be looking at
ways to chop the variables down to transmit less bytes, lets not try
to go the other way because the variable are nice and small now.
On Sep 29, 9:55 pm, Crashmatt <crash__m...@hotmail.com> wrote:
> I am starting to think that some structures will be larger than the
> CANbus 8 byte limit.
>
> An example is an absolute waypoint:
> Earth position to 1m is roughly 25bits each for lat and long.
> Rounding up you get to 4 bytes each, 8 bytes total.
> Then add a few bytes for waypoint settings.
>
> There are three alternatives I know of:
>
> 1. Use Pauls address based identifier and be very careful not to read
> or write parts of a variable.
important not to write parts of a variable. PIC has a disable
interrupts opcode so that's covered, what's not covered is switching
from using one set of variables to another. That needs to be done at a
higher level. As an example don't update the "live" way point, update
the next one and then switch to it so the high level AP code knows to
re-calculate targets, note flag changes and all those things that are
more or less one time events on way point switch rather than it having
to execute that code ever single navigation cycle.
>
> 2. Make CANbus capable of carrying larger structures
> Limits the address space
> Message buffers need to be implemented.
> Be very careful to make sure that the buffer is all filled from the
> same variable
>
your interested in getting lots of information. CAN message lengths
are set and will require silicon changes that are not going to happen.
It would have been nice to get 16 byte messages or have the DLC field
as power of 2 coding but it not. A simple trick to making sure you
have a coherent copy of a complex variable is to frame it on each side
with a key value eg when updating a way point you start building a
buffer by clearing all the contents, then transfer the actual values
into the middle part and then update both frame values - of course 0
should not be considered a valid frame "key" value in that case. This
would be a minimum for any control type messages. The actual key
values could be CRC values or hard coded and normal packet
verification should still happen.
> 3. Úse an addressing variable.
> A variable sets the currently selected structure (waypoint)
> Parts of the waypoint can be modified
> 64bit address space instead of ¨8bit.
> Extension of simple scheme
> Needs more bandwidth.
> Variables do not need to be arranged on a specific byte boundary.
>
that's 17 bit's of address. Variables will always need to be on word
boundaries if they are larger than one byte - that's part of the PIC
architecture.
> of which #3 is my favorite because it is the safest and is of least
> concern to the main code.
> It can also be implemented at a later date. We do not need to worry
> about it now.
>
> Matt
completely pointless, do it on the PC. The variables on the AP are by
definition as precise as they can be, making them smaller you lose
precision and making them bigger at best adds zero's that take longer
to transmit. The communication routine is always going to have to know
what its getting - trying to come up with a universal number format is
like trying to fit a square peg in a round hole and works for neither
the peg or hole. This gets mitigated at the source some because we try
to use all the precision available in the source variables but at a
fundamental level velocity is not equal to say altitude (the possible
ranges are just too different). If we were using some really high end
DSP were everything was 64bit floating point values we'd be looking at
ways to chop the variables down to transmit less bytes, lets not try
to go the other way because the variable are nice and small now.
Peter Hollands
Sep 30, 2010, 8:28:19 AM9/30/10
to rc-servo-...@googlegroups.com
That was good. Pete
UAVflightdirector
Sep 30, 2010, 2:27:53 PM9/30/10
to RC servo interface
Paul,
Agreed on your first comment on not writing the live data.
We would like the data to be good every time so we can rely on it.
It could be managed by not reading while the relevant processes are
running. A kind of data lockout.
A read buffering scheme would work but consume resources. Hopefully
it doesnt come to that.
Nice idea on the data framing. A couple of framing bits (start,stop)
in the identifier might do it.
Framing the whole dataset would be good. The end of the whole dataset
may be a useful trigger.
I am only thinking of the indirect addressing for large variables. We
may never have this problem.
I cant think of a good reason to have more than 256 of something on
the AP, nevermind 64bits worth.
I agree with you about doing the conversion at the PC or onboard linux
processor. The telemetry should speak autopilot, not groundstation.
Matt
Agreed on your first comment on not writing the live data.
We would like the data to be good every time so we can rely on it.
It could be managed by not reading while the relevant processes are
running. A kind of data lockout.
A read buffering scheme would work but consume resources. Hopefully
it doesnt come to that.
Nice idea on the data framing. A couple of framing bits (start,stop)
in the identifier might do it.
Framing the whole dataset would be good. The end of the whole dataset
may be a useful trigger.
I am only thinking of the indirect addressing for large variables. We
may never have this problem.
I cant think of a good reason to have more than 256 of something on
the AP, nevermind 64bits worth.
I agree with you about doing the conversion at the PC or onboard linux
processor. The telemetry should speak autopilot, not groundstation.
Matt
UAVflightdirector
Oct 1, 2010, 1:45:43 AM10/1/10
to RC servo interface
Paul,
I need your thoughts on some framing side effects:
If a data frame is longer than a message data frame, you need a buffer/
postbox for the whole data message.
Every item on CAN can be sending a message at the same time.
A postbox should track one message at a time.
Every item on the bus adds a buffer to every endpoint.
The protocol becomes a connected one.
or
Before transmitting a frame, the sender needs to request the use of a
postbox for itself.
My intention was to add sensor nodes to CANbus. I would very much
like to avoid adding more resource loading. Adding single tx and rx
buffers is ok. Adding a whole bunch and managing connections is not
so ok.
Is this correct? Have you a trick for this?
Regards Matt
On Sep 30, 8:27 pm, UAVflightdirector <uavflightdirec...@gmail.com>
wrote:
I need your thoughts on some framing side effects:
If a data frame is longer than a message data frame, you need a buffer/
postbox for the whole data message.
Every item on CAN can be sending a message at the same time.
A postbox should track one message at a time.
Every item on the bus adds a buffer to every endpoint.
The protocol becomes a connected one.
or
Before transmitting a frame, the sender needs to request the use of a
postbox for itself.
My intention was to add sensor nodes to CANbus. I would very much
like to avoid adding more resource loading. Adding single tx and rx
buffers is ok. Adding a whole bunch and managing connections is not
so ok.
Is this correct? Have you a trick for this?
Regards Matt
On Sep 30, 8:27 pm, UAVflightdirector <uavflightdirec...@gmail.com>
wrote:
Paul G.
Oct 1, 2010, 7:37:18 AM10/1/10
to RC servo interface
Matt,
Sensors that are not being actively used at servo rate don't matter at
all - whenever they update is fine - you don't need to worry for
anything you add. That goes for both sides of the data link. In
extreme cases it could result in some strange logs were the aircraft
could appear to jump a large distance and then back again. A good
trick for this is to build the packet twice and make sure they are
equal - its only possible for an error to exist if a) neglect to DSI
non-atomic reads AND b) get an update interrupt exactly between those
two (or more) non-atomic reads. It can only happen on multi-word
variables (there are next to none in the first place) and get
"unlucky", bottom line its not going to happen often, like once a year
kinda thing, and will have no real effect even if it does. Bear in
mind this only applies to single variables inside a larger structure,
getting a new latitude update with an old longitude update doesn't
matter at all as the difference between two latitude updates or two
longitude updates is going to be so small its well within the expected
error of the sensors anyway. This is another reason to do the memory
image interface - that "lag" is kept to a minimum and your always free
to transmit the data you have "as is" because you know the worst case
timing is at or below servo rate so there is not any "better" data to
be had.
Regards Paul
On Sep 30, 10:45 pm, UAVflightdirector <uavflightdirec...@gmail.com>
Sensors that are not being actively used at servo rate don't matter at
all - whenever they update is fine - you don't need to worry for
anything you add. That goes for both sides of the data link. In
extreme cases it could result in some strange logs were the aircraft
could appear to jump a large distance and then back again. A good
trick for this is to build the packet twice and make sure they are
equal - its only possible for an error to exist if a) neglect to DSI
non-atomic reads AND b) get an update interrupt exactly between those
two (or more) non-atomic reads. It can only happen on multi-word
variables (there are next to none in the first place) and get
"unlucky", bottom line its not going to happen often, like once a year
kinda thing, and will have no real effect even if it does. Bear in
mind this only applies to single variables inside a larger structure,
getting a new latitude update with an old longitude update doesn't
matter at all as the difference between two latitude updates or two
longitude updates is going to be so small its well within the expected
error of the sensors anyway. This is another reason to do the memory
image interface - that "lag" is kept to a minimum and your always free
to transmit the data you have "as is" because you know the worst case
timing is at or below servo rate so there is not any "better" data to
be had.
Regards Paul
On Sep 30, 10:45 pm, UAVflightdirector <uavflightdirec...@gmail.com>
Reply all
Reply to author
Forward
0 new messages