mksocfpga DE10-Nano encoder index issue (latency?)
81 views
Skip to first unread message
justin White
Sep 14, 2019, 2:04:42 PM9/14/19
to Machinekit
Doing a little testing on my hardware I noticed there is an issue with encoder indexes being missed while trying to count them. It's difficult in hal to see the index pin change state on an encoder with reasonable resolution because the change in state is very short. So I added the function in my GUI to count up the encoder index pulses because it's obviously more visible when a number increments up vs trying to catch a small blip in halshow or halcmd. I noticed the index pulses are missed spinning the encoder at anything other than a very slow speed. I'm not really sure what communication method mksocfpga uses between the fpga and the cpu but I figured I'd try running a few non-fp components in a 0.2ms base thread to see if it helped. Didn't really seem to help at all
I first tried this by routing the hm2<board>index-input hal pin into the updown component and sending the counts to a hal label in my gui. My first thought is that the state change is too short for the servo-thread to catch at 1ms, so I added the "edge" component to extend the length of the index-input on it's output but that didn't really help. The output of edge obviously only get's extended if it catches the input state change which it does no better than updown.
The conclusion I'm drawing is that the RT behavior of the CPU or the communication between the FPGA and CPU cores is too slow for whatever reason, that or there's some issue with the encoder module in mksocfpga's hm2. I'm using 3 channels of a quad differential receiver chip for each encoder input. There is no difference between the index channel and A-B channels hardware wise, and this is the same on 6 identical instances of encoder inputs. The only difference is that hm2 counts the A-B channels in the FPGA while the index is not. I haven't seen any indication of missed counts on the A-B channels counting 4000 edges in quadrature. I've messed with the hm2 encoder sample-frequency too which also did not help. The only thing that helped somewhat is running a 0.5ms servo-thread but it still missed quite a few index's, and this is with me spinning the encoder by hand.
I use this same model of encoder on a LinuxCNC machine with a Mesa 7i96 and again on a 7i76e and I've never really seen an index missed on those spindle motors at ~3000rpms. If this isn't an issue with the hm2 encoder module itself I'd expect to see the same issue with a normal GPIO input missing short/fast pulses but I would think that someone else would have noticed that issue by now?
Thoughts?
Michael Brown
Sep 14, 2019, 2:48:24 PM9/14/19
to Machinekit
I would find it easier to follow you line of thought in this bug report if you would state the purpose..
So why ?
And what are you trying to achieve ?
justin White
Sep 14, 2019, 3:25:02 PM9/14/19
to Machinekit
Purpose is simply to see that every index input is caught in hal, it's a test program.....there is no real purpose.
That being said, I think the test just caught onto something that I didn't realize might be a problem, that the index output of this particular encoder may be intermittently broken. I swapped the A-channel of the encoder into the index input of my board and it is much more reliably catching that pulse. Sooooo, when I get a chance I'll yank the encoder out of another machine and see what happens with that......could be a false alarm.
Michael Brown
Sep 14, 2019, 3:46:06 PM9/14/19
to Machinekit
Well in the fpga you have the encoder core running at around 100 Mhz, then you are attempting to count the indes pulses going into this HW core with a software cpu running at only 900 Mhz a bare 9 assembler ticks faster ...?
ce...@tuta.io
Sep 14, 2019, 5:01:39 PM9/14/19
to justin White, Machinekit
Sep 14, 2019, 20:04 by blaz...@gmail.com:
Stupid question, but how exactly are you counting the Z pulses in your GUI? Are taking into account the non-RT nature of the GUI?
I don't remember how exactly is the communication done in HostMot2, but I remember that you have to "compute" the index signal from A/B registers if you were to catch the sampling message (request and response from FPGA layer) outside the index occurrence.
Cern.
I don't remember how exactly is the communication done in HostMot2, but I remember that you have to "compute" the index signal from A/B registers if you were to catch the sampling message (request and response from FPGA layer) outside the index occurrence.
Cern.
justin White
Sep 14, 2019, 5:21:32 PM9/14/19
to Machinekit
The "GUI" isn't counting the pulses. As I said/ I'm using the hal component "updown" which is a RT component as any other. That component outputs an unsigned integer that I just push to a gladevcp hal label. So it's being "counted" at the servo thread rate of 1ms or the .5ms that I briefly tried. I don't do any python coding or anything like that other than a very simple Python file to load the GUI as you typically would. The whole GUI is Hal files with a gladevcp interface, all it's doing is printing the number on the updown count pin.
The 900MHz CPU in and of itself can't possibly be the issue. The HM2 encoder core runs at the same speed as it does on any of my Mesa ETH cards, and being ETH cards it means I use a Preempt-RT kernel on those PCs as well. An encoder spinning at 3000rpms and not missing a single pulse has that index pulse state in several orders of magnitude less FPGA cycles than me sitting here spinning an encoder by hand. A 2GHz x86 CPU with almost equal latency has far less opportunity to recognize that hm2 pin state than the Nano does at 900mhz with me spinning it by hand.
justin White
Sep 14, 2019, 5:40:14 PM9/14/19
to Machinekit
Actually this is a better example because I do the exact same thing on this gizmo but in real use. The encoder is driven by a 2" diameter roller and the web driving it is running at 900fpm. That means the roller is spinning at ~1700 RPMs. That machine is using a much more complicated hal file, it counts index pulses (using updown) then triggers the an output on the comparator component. It's the same encoder, a 7i96 and a LinuxCNC PC @ ~2ghz with not great latencies, but it never really misses an index count. That machine runs a fairly huge Python file but all the logic is done in hal.
Point is that all things being equal I probably shouldn't have an issue seeing an index pulse in the same manner at 1/100th the speed at 900MHz. But like I said, I do alot of monkeying around, maybe I damaged this encoder in some way.
justin White
Sep 14, 2019, 5:41:06 PM9/14/19
to Machinekit
Forgot the picture
justin White
Sep 15, 2019, 5:44:30 PM9/15/19
to Machinekit
I'm not 100% certain what effect it may have but I noticed the "hm2_XiXX.N.encoder.timer−number" pin is missing. According to the man pages, this pin should be available if hm2 has a dpll module, which it does unless the encoder module "does not support it". Stepgen does have the "timer-number" as it should.
the encoder.timer-number pin selects which hm2dpll tiimer to use to synchronize reads offset of the thread rate. Since it's unavailable I'd assume it would be the same as -1, or not used. Not something I use with my other machines but it looks like using a timer with a negative value helps account for latency issues in hm2 reads. Probably won't help with the index thing but the encoder velocity feedback tends to be all over the place below 100rpms, figured it may help with that.
On Saturday, September 14, 2019 at 5:41:06 PM UTC-4, justin White wrote:
Forgot the picture
ce...@tuta.io
Sep 18, 2019, 1:10:18 AM9/18/19
to justin White, Machinekit
Sep 14, 2019, 23:40 by blaz...@gmail.com:
> Actually this is a better example because I do the exact same thing on this gizmo but in real use. The encoder is driven by a 2" diameter roller and the web driving it is running at 900fpm. That means the roller is spinning at ~1700 RPMs. That machine is using a much more complicated hal file, it counts index pulses (using updown) then triggers the an output on the comparator component. It's the same encoder, a 7i96 and a LinuxCNC PC @ ~2ghz with not great latencies, but it never really misses an index count. That machine runs a fairly huge Python file but all the logic is done in hal.
>
> Point is that all things being equal I probably shouldn't have an issue seeing an index pulse in the same manner at 1/100th the speed at 900MHz. But like I said, I do alot of monkeying around, maybe I damaged this encoder in some way.
>
>
Actually, without seeing the code you are using to set-up your system, it is just abstract talking.
What I had in mind is to determine if the problem is in Machinekit-HAL proper, HostMot2 driver or the MESA FPGA firmware. The actual counting of encoder's signals (A/B/Z) happens inside the FPGA firmware core and outside the scope of Machinekit. Machinekit-HAL only with periodicity of read function loop inside a real-time task (for example the 1ms long so called base-period) asks about update and receive state of registers inside the FPGA at some time.
What is needed is to investigate if the SoC FPGA firmware is functioning properly and you are - for example - only counting something improperly in Machinekit-HAL (can be - as I said - the problem with non-RT access to HAL from GUI), or if the whole FPGA firmware is shot.
So, please, tell me exactly what you are doing inside the HAL.
Cern.
>
> On Saturday, September 14, 2019 at 5:21:32 PM UTC-4, justin White wrote:
>
>> The "GUI" isn't counting the pulses. As I said/ I'm using the hal component "updown" which is a RT component as any other. That component outputs an unsigned integer that I just push to a gladevcp hal label. So it's being "counted" at the servo thread rate of 1ms or the .5ms that I briefly tried. I don't do any python coding or anything like that other than a very simple Python file to load the GUI as you typically would. The whole GUI is Hal files with a gladevcp interface, all it's doing is printing the number on the updown count pin.
>>
>> The 900MHz CPU in and of itself can't possibly be the issue. The HM2 encoder core runs at the same speed as it does on any of my Mesa ETH cards, and being ETH cards it means I use a Preempt-RT kernel on those PCs as well. An encoder spinning at 3000rpms and not missing a single pulse has that index pulse state in several orders of magnitude less FPGA cycles than me sitting here spinning an encoder by hand. A 2GHz x86 CPU with almost equal latency has far less opportunity to recognize that hm2 pin state than the Nano does at 900mhz with me spinning it by hand.
>>
>> On Saturday, September 14, 2019 at 5:01:39 PM UTC-4, >> ce...@tuta.io <>>> wrote:
>>
>>> Sep 14, 2019, 20:04 by >>> blaz...@gmail.com <>>>> :
>>>
>>> > Doing a little testing on my hardware I noticed there is an issue with encoder indexes being missed while trying to count them. It's difficult in hal to see the index pin change state on an encoder with reasonable resolution because the change in state is very short. So I added the function in my GUI to count up the encoder index pulses because it's obviously more visible when a number increments up vs trying to catch a small blip in halshow or halcmd. I noticed the index pulses are missed spinning the encoder at anything other than a very slow speed. I'm not really sure what communication method mksocfpga uses between the fpga and the cpu but I figured I'd try running a few non-fp components in a 0.2ms base thread to see if it helped. Didn't really seem to help at all
>>> >
>>> > I first tried this by routing the hm2<board>index-input hal pin into the updown component and sending the counts to a hal label in my gui. My first thought is that the state change is too short for the servo-thread to catch at 1ms, so I added the "edge" component to extend the length of the index-input on it's output but that didn't really help. The output of edge obviously only get's extended if it catches the input state change which it does no better than updown.
>>> >
>>> > The conclusion I'm drawing is that the RT behavior of the CPU or the communication between the FPGA and CPU cores is too slow for whatever reason, that or there's some issue with the encoder module in mksocfpga's hm2. I'm using 3 channels of a quad differential receiver chip for each encoder input. There is no difference between the index channel and A-B channels hardware wise, and this is the same on 6 identical instances of encoder inputs. The only difference is that hm2 counts the A-B channels in the FPGA while the index is not. I haven't seen any indication of missed counts on the A-B channels counting 4000 edges in quadrature. I've messed with the hm2 encoder sample-frequency too which also did not help. The only thing that helped somewhat is running a 0.5ms servo-thread but it still missed quite a few index's, and this is with me spinning the encoder by hand.
>>> >
>>> > I use this same model of encoder on a LinuxCNC machine with a Mesa 7i96 and again on a 7i76e and I've never really seen an index missed on those spindle motors at ~3000rpms. If this isn't an issue with the hm2 encoder module itself I'd expect to see the same issue with a normal GPIO input missing short/fast pulses but I would think that someone else would have noticed that issue by now?
>>> >
>>> > Thoughts?
>>> >
>>> Stupid question, but how exactly are you counting the Z pulses in your GUI? Are taking into account the non-RT nature of the GUI?
>>>
>>> I don't remember how exactly is the communication done in HostMot2, but I remember that you have to "compute" the index signal from A/B registers if you were to catch the sampling message (request and response from FPGA layer) outside the index occurrence.
>>>
>>> Cern.
>>>
>
>
>
> --
> website: > http://www.machinekit.io <http://www.machinekit.io>> blog: > http://blog.machinekit.io <http://blog.machinekit.io>> github: > https://github.com/machinekit <https://github.com/machinekit>
> ---
> You received this message because you are subscribed to the Google Groups "Machinekit" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to > machinekit+...@googlegroups.com <mailto:machinekit+...@googlegroups.com>> .
> To view this discussion on the web visit > https://groups.google.com/d/msgid/machinekit/6ba00d4d-79e0-45bc-b7cb-0df4cfa5c085%40googlegroups.com <https://groups.google.com/d/msgid/machinekit/6ba00d4d-79e0-45bc-b7cb-0df4cfa5c085%40googlegroups.com?utm_medium=email&utm_source=footer>> .
>
> Actually this is a better example because I do the exact same thing on this gizmo but in real use. The encoder is driven by a 2" diameter roller and the web driving it is running at 900fpm. That means the roller is spinning at ~1700 RPMs. That machine is using a much more complicated hal file, it counts index pulses (using updown) then triggers the an output on the comparator component. It's the same encoder, a 7i96 and a LinuxCNC PC @ ~2ghz with not great latencies, but it never really misses an index count. That machine runs a fairly huge Python file but all the logic is done in hal.
>
> Point is that all things being equal I probably shouldn't have an issue seeing an index pulse in the same manner at 1/100th the speed at 900MHz. But like I said, I do alot of monkeying around, maybe I damaged this encoder in some way.
>
>
What I had in mind is to determine if the problem is in Machinekit-HAL proper, HostMot2 driver or the MESA FPGA firmware. The actual counting of encoder's signals (A/B/Z) happens inside the FPGA firmware core and outside the scope of Machinekit. Machinekit-HAL only with periodicity of read function loop inside a real-time task (for example the 1ms long so called base-period) asks about update and receive state of registers inside the FPGA at some time.
What is needed is to investigate if the SoC FPGA firmware is functioning properly and you are - for example - only counting something improperly in Machinekit-HAL (can be - as I said - the problem with non-RT access to HAL from GUI), or if the whole FPGA firmware is shot.
So, please, tell me exactly what you are doing inside the HAL.
Cern.
>
> On Saturday, September 14, 2019 at 5:21:32 PM UTC-4, justin White wrote:
>
>> The "GUI" isn't counting the pulses. As I said/ I'm using the hal component "updown" which is a RT component as any other. That component outputs an unsigned integer that I just push to a gladevcp hal label. So it's being "counted" at the servo thread rate of 1ms or the .5ms that I briefly tried. I don't do any python coding or anything like that other than a very simple Python file to load the GUI as you typically would. The whole GUI is Hal files with a gladevcp interface, all it's doing is printing the number on the updown count pin.
>>
>> The 900MHz CPU in and of itself can't possibly be the issue. The HM2 encoder core runs at the same speed as it does on any of my Mesa ETH cards, and being ETH cards it means I use a Preempt-RT kernel on those PCs as well. An encoder spinning at 3000rpms and not missing a single pulse has that index pulse state in several orders of magnitude less FPGA cycles than me sitting here spinning an encoder by hand. A 2GHz x86 CPU with almost equal latency has far less opportunity to recognize that hm2 pin state than the Nano does at 900mhz with me spinning it by hand.
>>
>> On Saturday, September 14, 2019 at 5:01:39 PM UTC-4, >> ce...@tuta.io <>>> wrote:
>>
>>> Sep 14, 2019, 20:04 by >>> blaz...@gmail.com <>>>> :
>>>
>>> > Doing a little testing on my hardware I noticed there is an issue with encoder indexes being missed while trying to count them. It's difficult in hal to see the index pin change state on an encoder with reasonable resolution because the change in state is very short. So I added the function in my GUI to count up the encoder index pulses because it's obviously more visible when a number increments up vs trying to catch a small blip in halshow or halcmd. I noticed the index pulses are missed spinning the encoder at anything other than a very slow speed. I'm not really sure what communication method mksocfpga uses between the fpga and the cpu but I figured I'd try running a few non-fp components in a 0.2ms base thread to see if it helped. Didn't really seem to help at all
>>> >
>>> > I first tried this by routing the hm2<board>index-input hal pin into the updown component and sending the counts to a hal label in my gui. My first thought is that the state change is too short for the servo-thread to catch at 1ms, so I added the "edge" component to extend the length of the index-input on it's output but that didn't really help. The output of edge obviously only get's extended if it catches the input state change which it does no better than updown.
>>> >
>>> > The conclusion I'm drawing is that the RT behavior of the CPU or the communication between the FPGA and CPU cores is too slow for whatever reason, that or there's some issue with the encoder module in mksocfpga's hm2. I'm using 3 channels of a quad differential receiver chip for each encoder input. There is no difference between the index channel and A-B channels hardware wise, and this is the same on 6 identical instances of encoder inputs. The only difference is that hm2 counts the A-B channels in the FPGA while the index is not. I haven't seen any indication of missed counts on the A-B channels counting 4000 edges in quadrature. I've messed with the hm2 encoder sample-frequency too which also did not help. The only thing that helped somewhat is running a 0.5ms servo-thread but it still missed quite a few index's, and this is with me spinning the encoder by hand.
>>> >
>>> > I use this same model of encoder on a LinuxCNC machine with a Mesa 7i96 and again on a 7i76e and I've never really seen an index missed on those spindle motors at ~3000rpms. If this isn't an issue with the hm2 encoder module itself I'd expect to see the same issue with a normal GPIO input missing short/fast pulses but I would think that someone else would have noticed that issue by now?
>>> >
>>> > Thoughts?
>>> >
>>> Stupid question, but how exactly are you counting the Z pulses in your GUI? Are taking into account the non-RT nature of the GUI?
>>>
>>> I don't remember how exactly is the communication done in HostMot2, but I remember that you have to "compute" the index signal from A/B registers if you were to catch the sampling message (request and response from FPGA layer) outside the index occurrence.
>>>
>>> Cern.
>>>
>
>
>
> website: > http://www.machinekit.io <http://www.machinekit.io>> blog: > http://blog.machinekit.io <http://blog.machinekit.io>> github: > https://github.com/machinekit <https://github.com/machinekit>
> ---
> You received this message because you are subscribed to the Google Groups "Machinekit" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to > machinekit+...@googlegroups.com <mailto:machinekit+...@googlegroups.com>> .
> To view this discussion on the web visit > https://groups.google.com/d/msgid/machinekit/6ba00d4d-79e0-45bc-b7cb-0df4cfa5c085%40googlegroups.com <https://groups.google.com/d/msgid/machinekit/6ba00d4d-79e0-45bc-b7cb-0df4cfa5c085%40googlegroups.com?utm_medium=email&utm_source=footer>> .
>
justin White
Sep 18, 2019, 6:35:35 AM9/18/19
to Machinekit
It's a moot point, but my display program is not counting anything. As I said the counting is being done by the RT-hal component "updown" which is as RT as any other component. The GUI is only displaying an integer, so if the encoder has spun 5 revolutions before a display refresh then there should be some increase of that integer,
It's a standard set of built in components so I figured it's simpler to explain it rather than post but these are the pertaining lines, the edge component is just a teset and can be omitted, doesn't change the fact that the pulse is too short for hal to catch:
That's insubstantial though because I did however mis-speak about how I do it on my other machine, I set it up a long time ago and it was rather difficult......probably for this reason. Catching the index-input pulse in hal is something you just can't do reliably so that's probably not a firmware issue. The actual use of getting it done is to set the index-enable pin true, this triggers a firmware reset of counts/position/and the index-enable pin itself on the next index event. so the means of counting, or just reliably "seeing" the index in hal is to constantly reset the index-enable pin and count how many times it goes false. That all seems to work as intended in MK, so I'll probably just leave it as is because resetting the counts/position on this test rig makes it pointless.
(main HAL)
net idx_edge_0 <= hm2_[HOSTMOT2](BOARD).0.encoder.00.input-indexnet idx_edge_0 => edge.0.innet idx_updn_0 <= edge.0.outnet idx_updn_0 => updown.0.countup
setp edge.0.out-width-ns 50000
(postGUI HAL)
net idx_cnt_lbl_0 updown.0.count => [DISPLAY](DISPLAY).hal_label_IDX_COUNT_0
That's insubstantial though because I did however mis-speak about how I do it on my other machine, I set it up a long time ago and it was rather difficult......probably for this reason. Catching the index-input pulse in hal is something you just can't do reliably so that's probably not a firmware issue. The actual use of getting it done is to set the index-enable pin true, this triggers a firmware reset of counts/position/and the index-enable pin itself on the next index event. so the means of counting, or just reliably "seeing" the index in hal is to constantly reset the index-enable pin and count how many times it goes false. That all seems to work as intended in MK, so I'll probably just leave it as is because resetting the counts/position on this test rig makes it pointless.
I wonder what kind of trouble it would be to add an index-count pin to hm2 encoder firmware. Thinking about it now that would certainly make things easier on the machine I do implement this type of thing on.
The velocity thing is still in question though, I do believe it's quite a bit rougher than a typical PC/Mesa setup. I would think that the fact that it's a slower CPU would only effect the read/update rate possible in the servo-thread, not the actual accuracy of the value. I asked about this and PCW's response was.....
Velocity is calculated in a delta-count/delta-time basis where delta-time is the time between the first and last of delta-counts (not the sample time) so it should be fairly independent of servo thread jitter, however if the timestamp clock calculations are not correct because the FPGA clock is not reported right or something like that, this would cause errors in the velocity calculations
So that's a bit beyond me but I get the gist of what he's saying. The accuracy should be fairly straightforward for any of the MK contributors to verify if they were to investigate. I'm personally going to build a more ridged and accurate test jig over the next few days. Like I said, I didn't expect to be able to spin a typical stepper motor up past 4k RPMS but since I can I need to make sure I can I'd have to get the runout out of the rig. I currently see a variation of about 15-20% @ 200rpm (~3.3rps). My 7i76e setup does get worse at slower speeds as well, but
> To unsubscribe from this group and stop receiving emails from it, send an email to > machi...@googlegroups.com <mailto:machinekit+unsub...@googlegroups.com>> .
Reply all
Reply to author
Forward
0 new messages