CoMuRoS

Running the Complete System

The CoMuRoS system requires four separate terminals to run all components. Follow these steps in order:

Terminal 1: Launch Simulation Environment

ros2 launch multi_robot multi_robot.launch.py

What it does:

• Starts Ignition Gazebo with the food court environment
• Spawns 3 robots with delays:
  • r1 (Cleaning Robot) at position (11.0, 0.0, 0.05) after 5 seconds
  • r2 (Delivery Robot) at position (0.0, 0.0, 0.05) after 15 seconds
  • r3 (Drone) at position (-1.0, 1.0, 0.05) after 25 seconds
• Sets up ROS-Gazebo bridge for /clock and /tf topics
• Opens RViz for multi-robot visualization

Terminal 2: Launch Robot LLM Interfaces

ros2 launch robot_llm robot_llms.launch.py

What it does:

• Starts LLM interface nodes for all robots:
  • cleaning_bot_llm_node - Interprets commands for cleaning robot
  • delivery_bot_llm_node - Interprets commands for delivery robot
  • drone_llm_node - Interprets commands for the drone
• Each node subscribes to task messages from the chat system
• Converts high-level natural language commands into robot-specific actions

Terminal 3: Launch Robot Control Services

ros2 launch robot_llm robot_services.launch.py

What it does:

• Starts position controller services for each robot:
  • cleaning_bot_position_controller (namespace: r1) - Holonomic controller
  • deliver_bot_position_controller (namespace: r2) - Holonomic controller
  • drone_position_controller (namespace: r3) - Drone flight controller
• Provides low-level motion control services
• Executes movement commands and trajectories

Terminal 4: Launch Chat Interface

ros2 launch chatty chat_system.launch.py

What it does:

• Starts the natural language chat interface GUI
• Launches chat manager for conversation history
• Starts task manager to convert natural language to robot tasks using LLMs
• Enables optional audio input/output (microphone and text-to-speech)
• Publishes time information for task scheduling

Optional parameters:

# Use specific LLM model (default: 4)
ros2 launch chatty chat_system.launch.py model:=1

# Use custom robot configuration
ros2 launch chatty chat_system.launch.py config_file:=robot_config_roscon_2025

# Enable voice input/output
ros2 launch chatty chat_system.launch.py enable_audio_input:=true enable_audio_output:=true

Example Commands

Once all four terminals are running, start commanding the multi-robot system:

• "Clean the place"
• "Serve coke to the child"
• "Which table is empty?"

Package Structure

The CoMuRoS robot packages are organized as follows:

CoMuRoS/robots/
├── multi_robot/          # Multi-robot simulation environment
│   ├── launch/          # multi_robot.launch.py
│   ├── env/             # Ignition resource paths
│   ├── models/          # Food court assets
│   ├── worlds/          # foodcourt_world.sdf
│   └── rviz/            # Multi-robot RViz config
│
├── x3_uav/              # X3 UAV drone packages
│   ├── x3_uav_description/
│   ├── x3_uav_ignition/
│   └── x3_uav_llm/
│
└── yahboom/             # Yahboom Rosmaster X3 ground robots
    ├── yahboom_rosmaster_description/
    ├── yahboom_rosmaster_gazebo/
    └── yahboom_llm/

Adding New Robots

Want to add your own robot to the system? Follow these steps:

Step 1: Add Robot to Simulation

Edit multi_robot/launch/multi_robot.launch.py and add your robot launch:

robot4_launch = TimerAction(
    period=30.0,
    actions=[
        IncludeLaunchDescription(
            PythonLaunchDescriptionSource(your_robot_launch_path),
            launch_arguments={
                'use_sim_time': use_sim_time,
                'robot_name': 'your_robot_name',
                'prefix': 'r4',
                'spawn_x': '5.0',
                'spawn_y': '5.0',
                'spawn_z': '0.05',
            }.items()
        )
    ]
)

Step 2: Create Robot Package Structure

your_robot_llm/
├── data/
│   ├── chat_history.txt
│   └── robot_task_history.txt
├── launch/
├── package.xml
├── setup.py
└── your_robot_llm/
    ├── __init__.py
    ├── your_robot_llm.py
    └── your_robot_services.py

Step 3: Add LLM Node

Edit robot_llm/launch/robot_llms.launch.py to include your robot's LLM interface node.

Step 4: Add Robot Service

Edit robot_llm/launch/robot_services.launch.py to include your robot's position controller service.

Step 5: Update Robot Configuration

Create or edit configuration files in chatty/config/ with robot capabilities and initial state.

Step 6: Build and Test

cd ~/ros2_ws
colcon build --packages-select your_robot_llm
source install/setup.bash

ros2 launch multi_robot multi_robot.launch.py
ros2 launch robot_llm robot_llms.launch.py
ros2 launch robot_llm robot_services.launch.py
ros2 launch chatty chat_system.launch.py