Robot Perception and Control

Simulators

Research oriented simulators

Popularized in robotics as sim2real research

• Gymnasium (OpenAI Gym): An API standard for single-agent reinforcement learning environments.
• MuJoCo:
• NVIDIA Issac: GPU accelerated simulation: Gym (depricated), Sim, and Orbit.
• RaiSim: physics engine for robotics and artificial intelligence research with easy to use C++ library.
• PyBullet:
• Drake (MIT): C++ toolbox started by the Robot Locomotion Group at the MIT and Toyota.
• Gazebo: part of ROS.

Game engines as simulators

General purpose (physics simulation, rendering, etc.) game engines:

• Unity3D (Unity Technologies): cross-platform game engine developed by Unity Technologies
  • Barracuda: neural network interface for Unity
• Unreal Engine (Epic Games): 3D computer graphics and game engines developed by Epic Games
• CryEngine (Crytek):
• Lumberyard (Amazon):
• Stingray (Autodesk):
• PhysX (Nvidia):

Describing a Robot

URDF

URDF (Unified Robot Description Format) is a standard format based on XML used to describe a robot model in simulators. A URDF models a robot as a tree structure composed of links and joints. Links represent the robot's physical parts, while joints define how these parts move relative to each other, specifying their spatial relationships.

Language in Simulator

Domain Randomization