CoMuRoS

📋 Setup

• ROS 2 Humble Hawksbill (LTS, recommended)
• Ubuntu 22.04 (Jammy Jellyfish)
• Python 3.10+ (default with Ubuntu 22.04)
• Git
• colcon build tool

Verify that ROS 2 is installed and sourced correctly:

# Check ROS 2 environment variables
printenv | grep -i ROS

# Expected output should include at least:
# ROS_DISTRO=humble
# ROS_VERSION=2
# ROS_PYTHON_VERSION=3
          

If ROS_DISTRO=humble is present, your ROS 2 environment is correctly set up and ready for CoMuRoS.

📥 Clone the Repository

Clone the anonymous repository into your ROS 2 workspace:

cd ~/ros2_ws/src
git clone https://anonymous.4open.science/r/CoMuRoS-0242/ comuros
cd ~/ros2_ws

⚙️ Install Dependencies

rosdep install --from-paths src --ignore-src -r -y
cd src/comuros
pip install -r requirements.txt

🚀 Build the Package

cd ~/ros2_ws
colcon build --symlink-install
source install/setup.bash
echo "source ~/ros2_ws/install/setup.bash" >> ~/.bashrc

🔐 Set API Keys

You need at least one API key (OpenAI, Gemini, or XAI/Grok). Add them temporarily:

export OPENAI_API_KEY="your_openai_key"
export GEMINI_API_KEY="your_gemini_key"
export XAI_API_KEY="your_grok_key"

Or permanently in ~/.bashrc.

🎯 Run the System

Recommended: launch everything at once:

ros2 launch chatty chat_system.launch.py

Advanced: run nodes individually for chat_manager, task_manager, and chat_gui.

🚀 Running the Complete System

The CoMuRoS system requires four separate terminals to run all components. Follow these steps in order:

Terminal 1: Launch Simulation Environment

ros2 launch multi_robot multi_robot.launch.py

What it does:

• Starts Ignition Gazebo with the food court environment
• Spawns 3 robots with delays:
  • r1 (Cleaning Robot) at position (11.0, 0.0, 0.05) after 5 seconds
  • r2 (Delivery Robot) at position (0.0, 0.0, 0.05) after 15 seconds
  • r3 (Drone) at position (-1.0, 1.0, 0.05) after 25 seconds
• Sets up ROS-Gazebo bridge for /clock and /tf topics
• Opens RViz for multi-robot visualization

Terminal 2: Launch Robot LLM Interfaces

ros2 launch robot_llm robot_llms.launch.py

What it does:

• Starts LLM interface nodes for all robots:
  • cleaning_bot_llm_node - Interprets commands for cleaning robot
  • delivery_bot_llm_node - Interprets commands for delivery robot
  • drone_llm_node - Interprets commands for the drone
• Each node subscribes to task messages from the chat system
• Converts high-level natural language commands into robot-specific actions

Terminal 3: Launch Robot Control Services

ros2 launch robot_llm robot_services.launch.py

What it does:

• Starts position controller services for each robot:
  • cleaning_bot_position_controller (namespace: r1) - Holonomic controller
  • deliver_bot_position_controller (namespace: r2) - Holonomic controller
  • drone_position_controller (namespace: r3) - Drone flight controller
• Provides low-level motion control services
• Executes movement commands and trajectories

Terminal 4: Launch Chat Interface

ros2 launch chatty chat_system.launch.py

What it does:

• Starts the natural language chat interface GUI
• Launches chat manager for conversation history
• Starts task manager to convert natural language to robot tasks using LLMs
• Enables optional audio input/output (microphone and text-to-speech)
• Publishes time information for task scheduling

Optional parameters:

# Use specific LLM model (default: 4)
ros2 launch chatty chat_system.launch.py model:=1

# Use custom robot configuration
ros2 launch chatty chat_system.launch.py config_file:=robot_config_roscon_2025

# Enable voice input/output
ros2 launch chatty chat_system.launch.py enable_audio_input:=true enable_audio_output:=true

💬 Example Commands

Once all four terminals are running start commanding and interacting with multirobot system, for example:

• "Clean the place"
• "Serve coke to the child"
• "Which table is empty?"

🏗️ Package Structure

The CoMuRoS robot packages are organized as follows:

CoMuRoS/robots/
├── multi_robot/          # Multi-robot simulation environment
│   ├── launch/          # multi_robot.launch.py
│   ├── env/             # Ignition resource paths
│   ├── models/          # Food court assets
│   ├── worlds/          # foodcourt_world.sdf
│   └── rviz/            # Multi-robot RViz config
│
├── x3_uav/              # X3 UAV drone packages
│   ├── x3_uav_description/
│   ├── x3_uav_ignition/
│   └── x3_uav_llm/
│
└── yahboom/             # Yahboom Rosmaster X3 ground robots
    ├── yahboom_rosmaster_description/
    ├── yahboom_rosmaster_gazebo/
    └── yahboom_llm/

🤖 Adding New Robots

Want to add your own robot to the system? Follow these steps:

Step 1: Add Robot to Simulation

Edit multi_robot/launch/multi_robot.launch.py and add your robot launch:

robot4_launch = TimerAction(
    period=30.0,  # Spawn delay in seconds
    actions=[
        IncludeLaunchDescription(
            PythonLaunchDescriptionSource(your_robot_launch_path),
            launch_arguments={
                'use_sim_time': use_sim_time,
                'robot_name': 'your_robot_name',
                'prefix': 'r4',
                'spawn_x': '5.0',
                'spawn_y': '5.0',
                'spawn_z': '0.05',
            }.items()
        )
    ]
)

Step 2: Create Robot Package Structure

Create a new package with this structure:

your_robot_llm/
├── data/
│   ├── chat_history.txt
│   └── robot_task_history.txt
├── launch/
├── package.xml
├── setup.py
└── your_robot_llm/
    ├── __init__.py
    ├── your_robot_llm.py
    └── your_robot_services.py

Step 3: Add LLM Node

Edit robot_llm/launch/robot_llms.launch.py to include your robot's LLM interface node.

Step 4: Add Robot Service

Edit robot_llm/launch/robot_services.launch.py to include your robot's position controller service.

Step 5: Update Robot Configuration

Create or edit configuration files in chatty/config/ with robot capabilities and initial state.

Step 6: Build and Test

cd ~/ros2_ws
colcon build --packages-select your_robot_llm
source install/setup.bash

# Test the new robot
ros2 launch multi_robot multi_robot.launch.py
ros2 launch robot_llm robot_llms.launch.py
ros2 launch robot_llm robot_services.launch.py
ros2 launch chatty chat_system.launch.py