TextMap Example

TextMap is an open-source project for leveraging textual information across navigation pipeline, including SLAM, path planning, and goal commands, all on CPU.

TextMap comprises four modules:

Package	Role
NavOCR	Selective detection & recognition of navigation-relevant text
TextMap	Text landmark mapping (Adding text landmarks to the map during SLAM)
text_nav_bridge	Goal setting through text commands & Interfaces for Nav2
text_nav_sim	Gazebo simulation with text signs and objects (based on turtlebot3_simulation)

Demo

Text Landmark Mapping

NavOCR detects text in RGB camera images. TextMap then lifts each detection into a 3D SLAM map.

Simulation mapping

Real-world mapping

Text-command Navigation (with Nav2)

Given a text command, text_nav_bridge finds the closest-matching landmark from the saved map, and sends it to Nav2 as a NavigateToPose action.

Examples

This repository provides examples of how to use TextMap.

Environment	SLAM package	Model format	Dataset	Coverage	Link
Gazebo simulation	slam_toolbox	ONNX	real-time simulation	Text mapping + Navigation	example
Gazebo simulation	slam_toolbox	OpenVINO	real-time simulation	Text mapping + Navigation	example
Real-world	rtabmap_ros	ONNX	Realsense D455 camera	Text mapping only*	example
Real-world	rtabmap_ros	ONNX	rosbag dataset	Text mapping only*	example

*Maps from rtabmap_ros are not stable for 2D path planning yet. We will add navigation support for rtabmap_ros soon.

---

Quick Start (Simulation)

End-to-end TextMap pipeline in Gazebo: build a map with text landmarks, then drive to them by name.

1. Prerequisites

This tutorial is primarily tested on ROS 2 humble, but it is designed to work across ROS 2 distributions.

# Make sure your ROS 2 environment is sourced.
echo $ROS_DISTRO

# Nav2 and the simulation SLAM backend
sudo apt install \
  ros-${ROS_DISTRO}-navigation2 ros-${ROS_DISTRO}-nav2-bringup \
  ros-${ROS_DISTRO}-slam-toolbox

# Gazebo and TurtleBot3
sudo apt install \
  ros-${ROS_DISTRO}-gazebo-ros-pkgs \
  ros-${ROS_DISTRO}-turtlebot3-gazebo \
  ros-${ROS_DISTRO}-turtlebot3-description \
  ros-${ROS_DISTRO}-turtlebot3-teleop

# (Optional) if you are using python venv
python3 -m venv ~/.venvs/textmap
source ~/.venvs/textmap/bin/activate
pip install --upgrade pip
pip install colcon-common-extensions

# Prerequisite for NavOCR (default: onnx)
pip install onnxruntime pyyaml opencv-python numpy

2. Clone and build

cd ~/ros2_ws/src  # your ros2 workspace

git clone git@github.com:kc-ml2/NavOCR.git
git clone git@github.com:kc-ml2/TextMap.git
git clone git@github.com:kc-ml2/text_nav_bridge.git
git clone git@github.com:kc-ml2/text_nav_sim.git

cd ..

source /opt/ros/${ROS_DISTRO}/setup.bash

rosdep update
rosdep install --from-paths src --ignore-src -r -y

colcon build

# (Optional) if you are using python venv
python -m colcon build --symlink-install

source install/setup.bash

3. Initialize Simulation

Gazebo simulation environment with text signs:

ros2 launch text_nav_sim simulation.launch.py

On the other terminal, run robot teleoperation:

ros2 run turtlebot3_teleop teleop_keyboard

4. SLAM & Text Landmark Mapping

On the other terminal, launch Textmap:

ros2 launch textmap textmap_slamtoolbox.launch.py \
  landmark_save_path:=~/map/sim_run/landmarks.yaml

NavOCR detects text in the camera images, slam_toolbox builds the SLAM map, and textmap anchors the detected text as 3D landmarks.

When you are done exploration and mapping, save the occupancy map before pressing Ctrl+C on the textmap terminal (slam_toolbox must still be running):

# Occupancy grid for AMCL (.pgm + .yaml)
ros2 run nav2_map_server map_saver_cli -f ~/map/sim_run/map \
  --ros-args -p map_subscribe_transient_local:=true

Then Ctrl+C the textmap terminal. textmap auto-exports landmarks.yaml on shutdown to the path given by landmark_save_path.

You should now have:

~/map/sim_run/
  map.pgm           # occupancy grid image
  map.yaml          # map_server metadata
  landmarks.yaml    # detected text landmarks with 3D positions

5. Navigation

Turn off text landmark mapping, and launch Nav2, amcl, and text_nav_bridge:

ros2 launch text_nav_bridge text_nav_sim.launch.py \
landmark_file:=$HOME/map/sim_run/landmarks.yaml \
map_yaml_file:=$HOME/map/sim_run/map.yaml

Send a text command — it will be matched against the landmarks and converted into a Nav2 goal:

ros2 topic pub --once /text_nav/command std_msgs/msg/String "data: 'Kitchen'"

Monitor progress:

ros2 topic echo /text_nav/status

Try other landmarks from the sim world: Bedroom, Bathroom, Office, Exit, Living Room, Laundry, Storage Room, Garage, Closet.

Troubleshooting

spawn_entity.py hangs or odom frame does not exist.

A previous Gazebo process is still alive. Kill it before relaunching:

killall -9 gzserver gzclient; pkill -9 -f ros2

FLAGS_enable_pir_* / oneDNN errors from NavOCR on Intel CPU.

Set these before launching:

export FLAGS_enable_pir_api=0
export FLAGS_enable_pir_in_executor=0

Conda (base) breaks spawn_entity.py.

Deactivate conda before launching. We recommend to use venv for compatibility wirh ROS 2.

License

Apache License 2.0. See LICENSE for the full text and per-package notices.