ROS Discussion Group, Tonight! Tuesday, Oct 14 @ 7pm PT

[camp .]

    It's Tuesday again! Whether you're a beginner or an expert, please join us if you're interested in ROS.  :-] 

Join Zoom Meeting

https://us06web.zoom.us/j/88983478865?pwd=bRdjh1GJpslwHJBANe46VCTlSewyNy.1

Meeting ID: 889 8347 8865, Passcode: 935511

Thanks,

Camp

[camp .]

Tonight! - cp

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/488525550.2859514.1760463938350%40mail.yahoo.com.

[James H Phelan]

ROS Team,

I'm  having a Nav2 issue.

After I launch my robot, slam localization, then Nav2, navigation launches ok, then complains:
[lifecycle_manager-10] [INFO] [1760489547.997986714] [lifecycle_manager_navigation]: Creating bond timer...
[controller_server-1] [INFO] [1760489613.643738544] [local_costmap.local_costmap]: Message Filter dropping message: frame 'laser_frame' at time 1760489612.625 for reason 'the timestamp on the message is earlier than all the data in the transform cache'
[controller_server-1] [INFO] [1760489613.735305673] [local_costmap.local_costmap]: Message Filter dropping message: frame 'laser_frame' at time 1760489612.731 for reason 'the timestamp on the message is earlier than all the data in the transform cache'

In Rviz2 the map looks good, the robot's in the right place, but if I try 2D Goal Pose or Nav2 Goal,

The robot avatar doesn't move and the physical robot doesn't try to move.  

Navigation fails to plot a path even though the way is clear in the Rviz map.

It will respond to joystick.

I wasted a 4 day weekend on a drunkard's walk with Claude-Sonnet-4.5 trying to troubleshoot this.

His summary is this:

Problem

LaserScan timestamps are ~2 seconds in the future, causing transform extrapolation errors in Nav2.

What We've Learned

1. The YDLiDAR SDK's scan.stamp field is initialized to 0 (not populated by the driver)
2. Using node->now() instead of scan.stamp still produces timestamps 2 seconds in the future
3. use_sim_time is correctly set to false on the ydlidar node
4. The system clock itself appears correct (verified with date)

Failed Fixes

1. Using scan.stamp directly → gave epoch time (1970)
2. Using node->now() → timestamps still 2 seconds ahead
3. Checked for /clock topic publisher → none exists
4. Verified use_sim_time parameter → already false

The root cause of why node->now() returns future time remains unidentified.

Perhaps the ROS Sig might give me their thoughts this evening.

Thanks!

RoverDoc

James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/49889669.3044821.1760487833656%40mail.yahoo.com.

[James H Phelan]

Team,

Thanks for the guidance last Tues' ROS meeting to not try to separate my launch file components like "legs of a frog" and expect it to jump.  But the problem persists when all are launched from stingray.launch.py.

But although it launches without errors, once it settles I get the persistent error:

[controller_server-7] [INFO] [1760915844.079459033] [local_costmap.local_costmap]: Message Filter dropping message: frame 'laser_frame' at time 1760915842.992 for reason 'the timestamp on the message is earlier than all the data in the transform cache'

The robot responds appropriately to the joystick, but Nav2 fails to even plot a path to Nav2 Goal, let alone attempt to move, instead giving the above timestamp error.

I wasted a 4 day weekend last week with Claude-Sonnet-4.5 on a "drunkard's walk" down many "rabbit holes" (to thoroughly homogenize metaphors) getting bad troubleshooting advice none of which came close to solving the problem.  If you want to suffer through that whole dialog, it's here:

https://poe.com/s/ZXUayH9LRfIBGBjHdaNk

Google led me to someone with a similar problem here:

https://answers.ros.org/question/393581/

but I was unable to recognize any actionable advice.  Perhaps someone more knowledgeable than I...?

Someone suggested a network timing issue, but all is running on the Pi5 on ROS2 jazzy on Ubuntu 24.04 except Rviz2 and joystick which are running on a Dell CORE i7 laptop also running Ubuntu 24.04 and ROS2 jazzy.

My github is:

https://github.com/JHPHELAN?tab=repositories

articubot_one for the navigation files and

articubot_one/robots/stingray/ for robot specific files

Any insight appreciated!

James

James H Phelan

"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

[Sergei Grichine]

James,

It is important that all launch files have the "use_sim_time" argument set to false for real robot (rqt might help seeing wrong timestamps, if any).

Having time synced using chrony helps if you run nodes on different machines.

There are parameters that help relax tolerance to timestamps difference of laser scans.

Look for missing frog's legs under the table, use meld to compare with my dev branch ;-)

I hope this helps.

Best Regards, 
-- Sergei
   

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/5eeac74e-4cd2-435e-8922-93eeb6ae8f56%40hal-pc.org.

[Marco Walther]

On 10/19/25 21:03, Sergei Grichine wrote:
> James,
>

> It is important that *all* launch files have the "use_sim_time" argument
> set to /false/ for real robot (/rqt/ might help seeing wrong timestamps,
> if any).

Yes!

>
> Having time synced using /chrony/ helps if you run nodes on different
> machines.

You need the times for all computers in your 'ROS domain' synchronized,
yes. But there are many different ways to handle the NTP protocol. Ntpd,
chrony or systemd-timesyncd (default for Ubuntu 24.04) can all do it as
long as they can find a (or even better a couple) consistent time source(s).

>
> There are parameters that help relax tolerance to timestamps difference
> of laser scans.

That might help to a certain extend, but you're basically loosening the
timing consistency requirements for a lot of the system. Usually not a
good idea.

-- Marco

>
> Look for missing frog's legs under the table, use /meld/ to compare with

> my dev branch ;-)
>
> I hope this helps.
>
>
>
> Best Regards,
> -- Sergei
>
> On Sun, Oct 19, 2025, 9:13 PM 'James H Phelan' via HomeBrew Robotics

> Club <hbrob...@googlegroups.com <mailto:hbrob...@googlegroups.com>>
> wrote:
>
> __

>
> Team,
>
> Thanks for the guidance last Tues' ROS meeting to not try to
> separate my launch file components like "legs of a frog" and expect
> it to jump.  But the problem persists when all are launched from

> stingray.launch.py <http://stingray.launch.py>.

>
> But although it launches without errors, once it settles I get the
> persistent error:
>

> *[controller_server-7] [INFO] [1760915844.079459033]

> [local_costmap.local_costmap]: Message Filter dropping message:
> frame 'laser_frame' at time 1760915842.992 for reason 'the timestamp

> on the message is earlier than all the data in the transform cache'*

>
> The robot responds appropriately to the joystick, but Nav2 fails to
> even plot a path to Nav2 Goal, let alone attempt to move, instead
> giving the above timestamp error.
>
> I wasted a 4 day weekend last week with Claude-Sonnet-4.5 on a
> "drunkard's walk" down many "rabbit holes" (to thoroughly homogenize
> metaphors) getting bad troubleshooting advice none of which came
> close to solving the problem.  If you want to suffer through that
> whole dialog, it's here:
>

> https://poe.com/s/ZXUayH9LRfIBGBjHdaNk <https://poe.com/s/
> ZXUayH9LRfIBGBjHdaNk>

>
> Google led me to someone with a similar problem here:
>

> https://answers.ros.org/question/393581/ <https://answers.ros.org/
> question/393581/>

>
> but I was unable to recognize any actionable advice.  Perhaps
> someone more knowledgeable than I...?
>
> Someone suggested a network timing issue, but all is running on the
> Pi5 on ROS2 jazzy on Ubuntu 24.04 except Rviz2 and joystick which
> are running on a Dell CORE i7 laptop also running Ubuntu 24.04 and
> ROS2 jazzy.
>
> My github is:
>

> https://github.com/JHPHELAN?tab=repositories <https://github.com/
> JHPHELAN?tab=repositories>

>
> articubot_one for the navigation files and
>
> articubot_one/robots/stingray/ for robot specific files
>
> Any insight appreciated!
>

> James
>
> James H Phelan
>
> "Nihil est sine ratione cur potius sit quam non sit"
> Leibniz
>
> On 10/14/2025 12:45 PM, camp . wrote:
>>     It's Tuesday again! Whether you're a beginner or an expert,
>> please join us if you're interested in ROS.  :-]
>>
>> Join Zoom Meeting
>> https://us06web.zoom.us/j/88983478865?

>> pwd=bRdjh1GJpslwHJBANe46VCTlSewyNy.1 <https://us06web.zoom.us/
>> j/88983478865?pwd=bRdjh1GJpslwHJBANe46VCTlSewyNy.1>

>> Meeting ID: 889 8347 8865, Passcode: 935511
>>
>> Thanks,
>> Camp
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "HomeBrew Robotics Club" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to hbrobotics+...@googlegroups.com

>> <mailto:hbrobotics+...@googlegroups.com>.

>> To view this discussion visit https://groups.google.com/d/msgid/
>> hbrobotics/488525550.2859514.1760463938350%40mail.yahoo.com

>> <https://groups.google.com/d/msgid/
>> hbrobotics/488525550.2859514.1760463938350%40mail.yahoo.com?
>> utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to hbrobotics+...@googlegroups.com

> <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/

> hbrobotics/5eeac74e-4cd2-435e-8922-93eeb6ae8f56%40hal-pc.org
> <https://groups.google.com/d/msgid/
> hbrobotics/5eeac74e-4cd2-435e-8922-93eeb6ae8f56%40hal-pc.org?
> utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "HomeBrew Robotics Club" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hbrobotics+...@googlegroups.com

> <mailto:hbrobotics+...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/
> hbrobotics/

> CA%2BKVXVO0On%2B68GwQquZ9H0KZE9ftVp2wiLY%2BM3NeXrk3T4jG4A%40mail.gmail.com <https://groups.google.com/d/msgid/hbrobotics/CA%2BKVXVO0On%2B68GwQquZ9H0KZE9ftVp2wiLY%2BM3NeXrk3T4jG4A%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Michael Wimble]

Keep in mind that the ‘use_sim_time’ this is usually handled by parameter substitution. This isn’t well documented anywhere that I noticed, like it was an afterthought, and so I see it done incorrectly now and then.

In many of your YAML files, you’ll see snippets like:

amcl: # See: https://assets.researchsquare.com/files/rs-225880/v1_covered.pdf?c=1631856303

ros__parameters:

use_sim_time: True

Well, you can go and put all of those use_sim_time values to True or False everywhere, and maybe have a set of YAML files for use with simulation and another set for non-simulation, but that’s not how it’s intended to be done. In launch files, you’ll mostly see, e.g.,

param_substitutions = {

'use_sim_time': use_sim_time,

'autostart': autostart}

configured_params = ParameterFile(

RewrittenYaml(

source_file=params_file,

root_key=namespace,

param_rewrites=param_substitutions,

convert_types=False, # CRITICAL: Don't convert types to preserve plugin strings

),

)

Node(

package='nav2_controller',

executable='controller_server',

output='screen',

respawn=use_respawn,

respawn_delay=2.0,

parameters=[configured_params],

arguments=['--ros-args', '--log-level', log_level],

remappings=remappings + [('cmd_vel', 'cmd_vel_nav')],

),

Where the call to RewrittenYaml replaces all of the use_sim_time values in the yaml with the launch value and then you pass the rewritten YAML to the node. Or, sometimes you will see the sneaker forms like:

urdf_as_xml = xacro.process_file(

xacro_file_path, mappings={"use_ros2_control": use_ros2_control_value, "sim_mode": use_sim_time.perform(context)}

).toxml()

For instance, my URDF files have conditionals in them (a lot of people probably aren’t aware that you can), and I have to pass in parameters to cause the URDF to be adjusted.

The point being, all the programs that care about use_sim_time are supposed to take that value via a parameter dictionary (in YAML form, usually) or as command-line arguments, and the launch files are supposed to fix it all up before launching nodes. Especially if you write your own Node or IncludeLaunchDescription objects in your launch file. You need to be aware whether the code needs to deal with use_sim_time and make sure you pass the value on in one of the expected ways, usually by either rewriting the YAML file before the node sees it, or by explicitly creating a dictionary with the expected key/value pairs when you instantiate the node.

If you have some nodes running with use_sim_time set to True, and others with it set to False, the nodes will typically get the wrong clock value to work with. Of course, it’s up to Node writers to understand that there are multiple clocks running in ROS 2 and to choose the correct one. That’s another thing that’s often taken for granted in the documentation. Setting user_sim_time doesn’t actually cause any magic at all to happen. The Nodes are supposed to read that parameter, if they care, and use it to select the correct clock source. That’s kind of why a lot of the items in navigation.yaml don’t have a use_time_time key at all—those nodes don’t care about the clock source. But that’s another discussion.

[Michael Wimble]

Oh, by the way, if it hasn’t been mentioned, just topic echo or even “topic echo —field header.timestamp “ you topics, particularly the lidar topic. Under simulation, all timestamps will have small second values, usually a few hundred or thousand seconds. When not simulating, the seconds should all be huge. If you find, say, the lidar has small seconds, it’s using simulation time instead of wall clock and bob’s your uncle; that’s where the launch problem is. In my scripts directory I think I wrote a script you can run that iterates over every published topic and measures the time skew. I was using it to measure sensor latency but your problem should really stick out.

> On Oct 19, 2025, at 10:45 PM, Marco Walther <marc...@gmail.com> wrote:
>
> On 10/19/25 21:03, Sergei Grichine wrote:
>> James,
>> It is important that *all* launch files have the "use_sim_time" argument set to /false/ for real robot (/rqt/ might help seeing wrong timestamps, if any).
>
> Yes!
>
>> Having time synced using /chrony/ helps if you run nodes on different machines.
>
> You need the times for all computers in your 'ROS domain' synchronized, yes. But there are many different ways to handle the NTP protocol. Ntpd, chrony or systemd-timesyncd (default for Ubuntu 24.04) can all do it as long as they can find a (or even better a couple) consistent time source(s).
>
>> There are parameters that help relax tolerance to timestamps difference of laser scans.
>
> That might help to a certain extend, but you're basically loosening the timing consistency requirements for a lot of the system. Usually not a good idea.
>
> -- Marco
>

> To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/14760a8f-c72c-4c26-b395-a266f9df7b02%40gmail.com.

[Alan Timm]

Hey guys, I'll be able to join tonight.  I have the freertos microros firmware 100% done and running on alfie.  everything's working but I'd appreciate feedback and solicit advice for next steps.
https://github.com/alansrobotlab2/alfiebot_ws/tree/main/src/alfie_firmware.

Alan

[Alan Timm]

Or tomorrow night.  Like Tuesday night.  Instead of tonight, which is Monday night.  Doh!

[Pito Salas]

Hi all

I found this thread interesting and possibly relevant to a problem I was having but I lost the through line. So I asked my friend to summarize it for me. No claims that this is useful but at least it’s less TLDR. Also a neat use case.

Pito

Here's a structured summary of the key questions, suggestions, and conclusions from the HomeBrew Robotics Club thread regarding the Nav2 timestamp error:

---

## 🧩 Core Issue Raised by James H. Phelan
- **Problem:** Nav2 fails to plot a path and drops messages with the error:
> “Message Filter dropping message: frame 'laser_frame' at time ... for reason 'the timestamp on the message is earlier than all the data in the transform cache'”
- **Setup:** ROS2 Jazzy on Ubuntu 24.04; Pi5 runs most nodes, Rviz2 and joystick run on a Dell i7 laptop.
- **GitHub:** [articubot_one repo](https://github.com/JHPHELAN?tab=repositories)
- **Request:** Seeking actionable advice after unproductive troubleshooting with Claude-Sonnet-4.5 and online forums.

---

## 🔍 Suggestions from the Group

### 🛠 Sergei Grichine
- **Launch files:** Ensure all launch files set `use_sim_time=false` for real hardware.
- **Debugging:** Use `rqt` to inspect timestamps.
- **Code comparison:** Use `/meld/` to compare with his dev branch.
- **Conclusion:** Timestamp misalignment likely due to simulation time being used unintentionally.

### ⏱ Marco Walther
- **Time sync:** All machines in the ROS domain must be synchronized.
- Recommends `chrony`, `ntpd`, or `systemd-timesyncd` (Ubuntu 24.04 default).
- **Parameter tuning:** Warns against loosening timestamp tolerances—may compromise system integrity.
- **Conclusion:** Time synchronization is critical; relaxing constraints is risky.

### 📡 Michael Wimble
- **Timestamp inspection:** Use `ros2 topic echo -f header.timestamp` on topics like LIDAR.
- Small timestamp values → simulation time.
- Large values → wall clock time.
- **Tooling:** Mentions a custom script to measure time skew across topics.
- **Conclusion:** Timestamp mismatch likely due to simulation time being active on some nodes.

### 🤖 Alan Timm
- **Status update:** Shared progress on FreeRTOS micro-ROS firmware for “Alfie” robot.
- **Request:** Open to feedback and next-step suggestions.
- **GitHub:** [alfie_firmware repo](https://github.com/alansrobotlab2/alfiebot_ws/tree/main/src/alfie_firmware)

---

## ✅ Conclusions and Actionable Steps

| Recommendation | Contributor | Purpose |
|----------------|-------------|---------|
| Set `use_sim_time=false` in all launch files | Sergei | Prevent simulation time from causing transform cache errors |
| Inspect timestamps with `topic echo -f header.timestamp` | Michael | Identify which topics are using simulation time |
| Sync clocks across machines using NTP tools | Marco | Ensure consistent time across ROS nodes |
| Use `rqt` and `/meld/` for debugging | Sergei | Visualize and compare configurations |
| Run time skew script | Michael | Detect latency and timestamp misalignment |

---

Let me know if you'd like this formatted for email or turned into a troubleshooting checklist.
Best,

Pito

Boston Robot Hackers &&
Comp. Sci Faculty, Brandeis University (Emeritus)

> To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/682ED9EE-7D0D-4B10-A21F-4971BCFC528E%40gmail.com.

[Pito Salas]

I wrote too soon… A

fter reading the summary I have this question that is related somewhat. I get that error too sometimes, or something very similar. I judged that the clock on the VMWare hosted ROS2 was very close to the Raspi RT Ubuntu hosted ROS2. Very close but my tools are not sufficient to tell exactly. I think there could be a variance of maybe 300 ms. I am using Crony but maybe it’s not charny-ing perfectly.

My questions:

- do we think 300 ms off is enough to trigger the error?
- is there a good way to figure it out better?

Maybe a node that simply publishes a topic from the Raspi containing its local time and a node that subscribes to it on the VMWare node and then compares the difference. But that must exist already?

Best,

Pito

Boston Robot Hackers &&
Comp. Sci Faculty, Brandeis University (Emeritus)

[Michael Wimble]

That was actually pretty slick.

[Michael Wimble]

By default, I see to recall that default navigation.yaml mostly defaults tolerance to about 300 ms, so that’s definitely a threshold. A slow robot can work with worse skew, a faster robot won’t do well with this. the network sime syncs usually are much, much more precise in alignment. In my https://github.com/wimblerobotics/Sigyn/blob/main/scripts/py_scripts/show_time_skew.py you can:

# For a laser scan topic

ros2 run py_scripts show_time_skew --ros-args -p topic:=/scan

I haven’t looked at this in a long time but it might generate a nice report of the actual skew of the topic time stamp vs current wall clock time.

[Michael Wimble]

Also, I’ll mention, though it’s not likely your problem, that network time syncs to happen instantaneously, they have to sneek up to synchronization. If you change the clock on a system, bad, bad things can happen if it’s not done correctly. I’m not sure what happens if you system has a clock ahead of the master as attempting to go backwards is likely to be pretty hard. But, this probably is only an issue if you boot up a system and immediately rely on clocks being synchronized. This if from ancient memory, back when I ran kerosene computers so maybe the algorithms have changed.

[agmiller]

[controller_server-1] [INFO] [1760489613.735305673] [local_costmap.local_costmap]: Message Filter dropping message: frame 'laser_frame' at time 1760489612.731 

Boiling this frog down :), what are the implications of the above message?   

It seems that some component of the system was slow to produce/transfer/process a laser scan, and the local cost map had already been updated using other more recent data sources.   The timestamp in the warning seems actual, just delayed. 

The local cost mapper should reject the delayed scans, send this warning, and quickly move on to processing the next scan.    If the problem continues, this seems to be a queuing, transport, or performance problem.  

Do we have multiple scan data sources that the mapper is processing?  Lidar and Sonar?  Stamped and Unstamped?  Transformed and Raw?  Something else (besides the Lidar scan) that would trigger a cost map update?   Something that could be issuing unreal timestamp values?

- A - 

[James H Phelan]

Pito, 

Which "friend" did you ask?

James H Phelan
"Nihil est sine ratione cur potius sit quam non sit"
Leibniz

[Pito Salas]

In that case it was Claude, where I have a $20 / month subscription. But I also use copilot when Claude is not available. In general I like claude better for coding. I am. It sure what level copilot I have because I get it via GitHub, through a free package for professors. So I don’t pay but it is somewhat better than the free level of copilot. 

Best,

Pito Salas

Boston Robot Hackers &&

Computer Science Faculty, Brandeis University

On Oct 21, 2025, at 9:27 PM, 'James H Phelan' via HomeBrew Robotics Club <hbrob...@googlegroups.com> wrote:



To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/c74570ae-1f58-4614-b9ee-a10971e742dc%40hal-pc.org.