Safe Planning and Policy Optimization via World Model Learning

---

Overview

SPOWL (Safe Planning and Policy Optimization via World Model Learning) is a framework for safe reinforcement learning that unifies world model learning and policy optimization. It leverages latent-space dynamics modeling and constrained optimization to achieve safe and efficient learning in complex continuous control environments.

---

Requirements

• Python
• miniconda/conda

---

Installation

Get started with SPOWL:

1. Create a conda environment:

   conda create -n spowl python==3.10

2. Activate the environment:

   conda activate spowl

3. Install Safety Gymnasium

   wget https://github.com/PKU-Alignment/safety-gymnasium/archive/refs/heads/main.zip
   unzip main.zip
   rm -rf main.zip
   pip install -e safety-gymnasium-main

4. Install jax:

   pip install --no-cache-dir --upgrade pip
   pip install --no-cache-dir --upgrade "jax[cuda12]"

5. Install other requirements:

   pip install --no-cache-dir hydra-core tabulate wandb tqdm moviepy equinox optax

6. Install for 'osmesa':

   conda install -c conda-forge mesalib

7. Fix dependencies:

   pip install --no-cache-dir gymnasium-robotics==1.2.3 numpy==1.25.0

---

Usage

Run the training script to display all available options and configurations with:

python train.py --help

Run the training script to train default SPOWL configuration:

python train.py

---

SPOWL in some tasks

Point Goal 1

Point Goal 2

Point Button 1

Point Push 1

Car Goal 1

Doggo Goal 1

Ant Goal 1

---

Citation

If you use SPOWL in your research, please cite:

@article{latyshev2025spowl,
  title={Safe Planning and Policy Optimization via World Model Learning},
  author={Latyshev, Artem and Gorbov, Gregory and Panov, Aleksandr I.},
  journal={arXiv preprint arXiv:2506.04828},
  year={2025}
}