Skip to content

Sudden Unregistering of Remote Node (Unsync Node Register) in Multi-Machine Setup #201

Description

@KhairulM

Setup

I have 2 machines running ubuntu 20.04 and ROS Noetic on the same network:

  • Machine 1 is for running a gazebo simulation (Alias: "FMS")
  • Machine 2 is for spawning the robot in the simulation and running the navigation stack

I've configured the /etc/hosts of both of the machine and multicasting have been enabled (/proc/sys/net/ipv4/ip_forward set to 1 and /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts set to 0)

This is my launch file for Machine 1:

<launch>
	<arg name="gui" default="false"/>
	
	<node name="master_discovery" pkg="fkie_master_discovery" type="master_discovery" output="screen">
		<rosparam param="robot_hosts">['ROBOT1', 'ROBOT2']</rosparam>
	</node>
	<include file="$(find gazebo_ros)/launch/empty_world.launch">
	    <arg name="world_name" value="$(find turtlebot3_gazebo)/worlds/turtlebot3_house.world"/>
	    <arg name="paused" value="false"/>
	    <arg name="use_sim_time" value="true"/>
	    <arg name="gui" value="$(arg gui)"/>
	    <arg name="headless" value="false"/>
	    <arg name="debug" value="false"/>
            <arg name="verbose" value="true"/>
	</include>
</launch>

This is my launch file for Machine 2:

<launch>
        <arg name="model" default="waffle_pi" doc="model type [burger, waffle, waffle_pi]"/>
        <arg name="robot_name" default="robot1" doc="name of the robot (must be unique and does not contain whitespaces)"/>
        <arg name="x_pos" default="-4.0"/>
        <arg name="y_pos" default="1.0"/>
        <arg name="z_pos" default="0.0"/>
        
        <node name="master_discovery" pkg="fkie_master_discovery" type="master_discovery" output="screen">
		<!-- <rosparam param="robot_hosts">['FMS']</rosparam> -->
		<rosparam param="heartbeat_hz">2</rosparam>
		<!-- <rosparam param="send_mcast">False</rosparam> -->
		<!-- <rosparam param="listen_mcast">False</rosparam> -->
	</node>
        <node name="master_sync" pkg="fkie_master_sync" type="master_sync" output="screen">
                <rosparam param="ignore_hosts"> ['ROBOT2'] </rosparam>
                <rosparam param="sync_hosts"> ['FMS'] </rosparam>
		<!-- <rosparam param="ignore_topics"> ['*theora*', '*compressedDepth*'] </rosparam> -->
		<!-- <rosparam param="resync_on_reconnect">False</rosparam> -->
        </node>

        <param name="/use_sim_time" value="true"/>
        
        <group ns="$(arg robot_name)">
                <param name="robot_description" command="$(find xacro)/xacro --inorder '$(find turtlebot3_description)/urdf/turtlebot3_$(arg model).urdf.xacro' robot_name:=$(arg robot_name)" />
                <node name="spawn_urdf" pkg="gazebo_ros" type="spawn_model" args="-urdf -model $(arg robot_name)_turtlebot3 -x $(arg x_pos) -y $(arg y_pos) -z $(arg z_pos) -param robot_description -robot_namespace $(arg robot_name)" output="screen" />
                <node name="robot_state_publisher" pkg="robot_state_publisher" type="robot_state_publisher" output="screen">
                        <param name="publish_frequency" type="double" value="50.0"/>
                </node>
                <node name="base_link_to_laser" pkg="tf" type="static_transform_publisher" args="-0.032 0 0.1397 0 0 0 base_link base_scan 100"/>

                <!-- tranformations relay -->
                <node name="prefixed_odom_relay" pkg="tf" type="static_transform_publisher" args="0 0 0 0 0 0 odom $(arg robot_name)/odom 100"/>
                <node name="prefixed_base_footprint_relay" pkg="tf" type="static_transform_publisher" args="0 0 0 0 0 0 $(arg robot_name)/base_footprint base_footprint 100"/>
        </group>
	

	<!-- topic relay nodes -->
	<node name="$(arg robot_name)_cmd_vel_relay" pkg="topic_tools" type="relay" args="/cmd_vel /$(arg robot_name)/cmd_vel" output="screen" />
	<node name="$(arg robot_name)_imu_relay" pkg="topic_tools" type="relay" args="/$(arg robot_name)/imu /imu" output="screen" />
	<node name="$(arg robot_name)_joint_states_relay" pkg="topic_tools" type="relay" args="/$(arg robot_name)/joint_states /joint_states" output="screen"/>
	<node name="$(arg robot_name)_odom_relay" pkg="topic_tools" type="relay" args="/$(arg robot_name)/odom /odom" output="screen" />
	<node name="$(arg robot_name)_scan_relay" pkg="topic_tools" type="relay" args="/$(arg robot_name)/scan /scan" output="screen"/>
</launch>

I spawned a turtlebot3 model that I have edited so the topics and tf are prefixed properly and the local nodes on Machine 2 can communicate using regular topics/services name such as /odom, /scan, etc

Problem

When I run the navigation stack (this is a custom navigation stack we develop) in Machine 2, sometimes this log appears Unregistering /gazebo, its pretty inconsistent to when the log appears after running the navigation stack. But when I tried to get the info of the gazebo node (rosnode info /gazebo) in Machine 2 I got this
WhatsApp Image 2024-09-03 at 11 42 54 AM
The publications and services is empty on Machine 2, but on Machine 1 the gazebo node still has the correct publications and services and still publishing the correct topic

What could cause this error?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions