RoboHarm: Do Frontier Robot Policies Refuse Unsafe Instructions?

The RoboHarm project aims to develop a set of evaluation metrics to assess the safety of robots that can cause harm, such as those used in search and rescue or autonomous vehicles. Researchers have developed a set of 31 scenarios to test robot policies that refuse unsafe instructions, with 21 of them being new additions. The scenarios cover various robotic systems, including robotic arms, drones, and autonomous vehicles. The evaluation metrics include measures of safety, effectiveness, and robustness, and the researchers have also proposed a benchmarking framework to facilitate the evaluation of robot safety. The goal is to ensure that robots are designed to refuse unsafe instructions and operate safely in uncertain or unpredictable environments.