【机器人】复现 DualMap 具身导航 | 动态场景 | 双地图机制
本文复现DualMap ，它支持机器人通过自然语言查询在动态变化的环境中理解和导航
特点：双地图导航，结合全局抽象地图进行高层次快速选择，以及局部具体地图进行精确目标定位，有效管理和更新环境中的动态变化
代码地址：https://github.com/Eku127/DualMap
示例效果：
1、下载工程代码
执行下面的命令，拉取github的代码：
git clone https://github.com/Eku127/DualMap.git cd DualMap/3rdparty/ git clone https://github.com/apple/ml-mobileclip.git cd ml-mobileclip && git checkout 1140b8d打印信息：
$ git clone https://github.com/Eku127/DualMap.git正克隆到 'DualMap'...
remote: Enumerating objects: 493, done.
remote: Counting objects: 100% (63/63), done.
remote: Compressing objects: 100% (30/30), done.
remote: Total 493 (delta 43), reused 33 (delta 33), pack-reused 430 (from 2)
接收对象中: 100% (493/493), 86.42 MiB | 14.36 MiB/s, 完成.
处理 delta 中: 100% (165/165), 完成.
.....
$ cd ml-mobileclip && git checkout 1140b8d
注意：正在切换到 '1140b8d'。
您正处于分离头指针状态。您可以查看、做试验性的修改及提交，并且您可以在切换
回一个分支时，丢弃在此状态下所做的提交而不对分支造成影响。
如果您想要通过创建分支来保留在此状态下所做的提交，您可以通过在 switch 命令
中添加参数 -c 来实现（现在或稍后）。例如：
git switch -c <新分支名>
或者撤销此操作：
git switch -
通过将配置变量 advice.detachedHead 设置为 false 来关闭此建议
HEAD 目前位于 1140b8d MobileCLIP iOS App.
2、创建Conda环境
通过environment.yml配置文件来创建Conda环境，名字为dualmap_env，python版本为3.10；
environment.yml配置文件，如下所示：
name: dualmap_env channels: - nvidia - pytorch - conda-forge - defaults dependencies: - python=3.10 - cmake - pip - faiss-cpu=1.9.0 - mkl - blas=1.0=mkl - pip: - tyro - open_clip_torch - wandb - h5py - openai - hydra-core - distinctipy - ultralytics==8.3.103 - dill - supervision==0.25.1 - open3d - imageio - natsort - kornia - rerun-sdk==0.22.1 - record3d==1.4.1 - pyliblzfse - pypng - tabulate - pympler - plyfile - "numpy<2.0" - git+https://github.com/ultralytics/CLIP.git 执行下面命令进行创建：
conda env create -f environment.yml等待相关的依赖库安装完成，再进入dualmap_env环境：
conda activate dualmap_env3、安装torch和flash-attn
我们指定安装torch版本为2.5.1+cu118，执行下面命令进行安装：
pip install torch==2.5.1+cu118 torchvision==0.20.1+cu118 --index-url \ https://download.pytorch.org/whl/cu118等待安装完成～
Successfully installed nvidia-cublas-cu11-11.11.3.6 nvidia-cuda-cupti-cu11-11.8.87 nvidia-cuda-nvrtc-cu11-11.8.89 nvidia-cuda-runtime-cu11-11.8.89 nvidia-cudnn-cu11-9.1.0.70 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.3.0.86 nvidia-cusolver-cu11-11.4.1.48 nvidia-cusparse-cu11-11.7.5.86 nvidia-nccl-cu11-2.21.5 nvidia-nvtx-cu11-11.8.86 sympy-1.13.1 torch-2.5.1+cu118 torchvision-0.20.1+cu118 triton-3.1.0
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap$
再安装flash-attn版本为2.5.9.post1
pip install flash-attn==2.5.9.post1 --no-deps 4、安装MobileCLIP
执行下面命令进行安装：
cd 3rdparty/mobileclip pip install -e . --no-deps cd ../..打印信息：
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap$
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap$ cd 3rdparty/mobileclip
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap/3rdparty/mobileclip$ pip install -e . --no-deps
Obtaining file:///home/lgp/2025_project/DualMap/3rdparty/mobileclip
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: mobileclip
Building editable for mobileclip (pyproject.toml) ... done
Created wheel for mobileclip: filename=mobileclip-0.1.0-0.editable-py3-none-any.whl size=5659 sha256=d77226ca58cd1636890ac92db7d575e83673fc29796d9f2e2ba70f4af05d268c
Stored in directory: /tmp/pip-ephem-wheel-cache-1jfolsbp/wheels/26/a9/56/6bc77673ac3574dff1d51b7f0096f3f85b8a8ff6f8ff6d73d3
Successfully built mobileclip
Installing collected packages: mobileclip
Successfully installed mobileclip-0.1.0
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap/3rdparty/mobileclip$
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap/3rdparty/mobileclip$ cd ../..
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap$
5、下载Replica数据集
首先创建一个目录dataset，用于存放数据集，然后进行目录
mkdir dataset && cd dataset执行下面命令下载Replica数据集：
wget https://cvg-data.inf.ethz.ch/nice-slam/data/Replica.zip得到的Replica.zip大约12.5G左右，然后解压文件后的目录结构为：
dataset/
├── Replica/
│ ├── office0/
│ │ ├── results/ # RGB-D frames (depth + RGB)
│ │ └── traj.txt # Trajectory file
│ ├── office1/
│ ├── ...
│ └── room2/
│
6、构建地图
执行下面命令进行模型推理：
python -m applications.runner_dataset首次运行时会联网下载一些权重，包括：open_clip_model.safetensors、yolov8l-world.pt、mobile_sam.pt、FastSAM-s.pt
运行的打印信息：
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap$ python -m applications.runner_dataset
[2025-11-16 20:53:21,457] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
UserWarning: WARNING: failed to get cudart_version from onnxruntime build info.
2025-11-16 20:53:22,051 - WARNING - [Runner Dataset]
============================================================
Log Path : /home/lgp/2025_project/DualMap/output/map_results/log/log_20251116_205322.log
Output Dir : ./output/map_results
Map Save Dir : ./output/map_results/replica_office1/map
Class List Path : ./config/class_list/gpt_indoor_general.txt
Use FastSAM for OV? : True
Running Concrete Map Only? : True
Save Concrete Map? : True
Save Global Map? : False
Use Preload Global Map? : False
Use Rerun for Visualization? : True
============================================================
FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
Processing: 100%|█████████████████████████████████████████████| 200/200 [00:42<00:00, 4.65it/s]
Done
看到可视化界面，在逐渐构建详细3D环境信息：
运行过程：
最终完成构建：
如果想换其他场景，可以修改config/base_config.yaml文件
具体是修改scene_id变量，Replica数据集支持：office0、office1、office2、office3、room0、room1等
# ============================================================ # DATASET CONFIG # ============================================================ ## Replica Dataset dataset_name: replica scene_id: office1 dataset_path: dataset/Replica dataset_conf_path: config/data_config/dataset/replica.yaml dataset_gt_path: dataset/Replica-Dataset/Replica_original # ============================================================ # OUTPUT CONFIG # ============================================================ # output directory output_path: ./output/map_results # ============================================================ # LOGGING CONFIG # ============================================================ # logging config logging_config: ./config/support_config/logging_config.yaml其他场景的运行效果1：
其他场景的运行效果2：
7、模型推理——导航查找物体
然后我们需要基于构建好的地图信息，来导航查找物体；
python -m applications.offline_local_map_query打印信息：
(dualmap_env) lgp@lgp-MS-7E07:~/2025_project/DualMap$ python -m applications.offline_local_map_query
Loading classes id --> colors from: ./config/class_list/gpt_indoor_general_id_colors.json
Loading classes id --> names from: ./config/class_list/gpt_indoor_general.txt
Loading saved obj results from: ./output/map_results/replica_room0/map
Loading viewpoint from: ./output/map_results/replica_room0/map/viewpoint.json
Successfully loaded 58 objects
Loading CLIP model
[2025-11-16 21:42:16,868][root][INFO] - Parsing model identifier. Schema: None, Identifier: MobileCLIP-S2
[2025-11-16 21:42:16,868][root][INFO] - Loaded built-in MobileCLIP-S2 model config.
[2025-11-16 21:42:17,485][httpx][INFO] - HTTP Request: HEAD https://hf-mirror.com/apple/MobileCLIP-S2-OpenCLIP/resolve/main/open_clip_model.safetensors "HTTP/1.1 302 Found"
[2025-11-16 21:42:17,488][root][INFO] - Instantiating model architecture: CustomTextCLIP
[2025-11-16 21:42:18,065][root][INFO] - Loading full pretrained weights from: /home/lgp/.cache/huggingface/hub/models--apple--MobileCLIP-S2-OpenCLIP/snapshots/8e8a808316aeb7c24d0400e1cf8f74b6937832aa/open_clip_model.safetensors
[2025-11-16 21:42:18,115][root][INFO] - Final image preprocessing configuration set: {'size': (256, 256), 'mode': 'RGB', 'mean': (0.0, 0.0, 0.0), 'std': (1.0, 1.0, 1.0), 'interpolation': 'bilinear', 'resize_mode': 'shortest', 'fill_color': 0}
[2025-11-16 21:42:18,115][root][INFO] - Model MobileCLIP-S2 creation process complete.
/home/lgp/anaconda3/envs/dualmap_env/lib/python3.10/site-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
[2025-11-16 21:42:18,535][root][INFO] - Parsing tokenizer identifier. Schema: None, Identifier: MobileCLIP-S2
[2025-11-16 21:42:18,535][root][INFO] - Attempting to load config from built-in: MobileCLIP-S2
[2025-11-16 21:42:18,535][root][INFO] - Using default SimpleTokenizer.
Done initializing CLIP model.
Obj Map length: 58
Keybindings:
Q - Quit the application
R - Display the point cloud with RGB colors
C - Display the point cloud with semantic colors
F - Enter a query to find top similarity objects
H - Display this help message
N - Highlight objects based on previous query results
M - Colored objects based on previous query results
S - Save the current viewpoint
Press the corresponding key to perform the action.
看到可视化界面：
点击可视化界面，按下C键，查看三维点云的可视化：
点击可视化界面，按下F键，输入查找目标，比如：desk lamp（台灯）
Enter your query: desk lamp
Top 5 similar objects:
1. No. 17 lamp: 0.546
能看到找到了：
也可以输入中文的，点击可视化界面，按下F键，输入查找目标，比如：花
Enter your query: 花
Top 5 similar objects:
1. No. 35 unknown: 0.465
能看到找到了：
分享完成～
