RobotPerf: An Open-Source, Vendor-Agnostic, Benchmarking Suite for Evaluating Robotics Computing System Performance

Víctor Mayoral-Vilches,Jason Jabbour,Yu-Shun Hsiao,Zishen Wan,Martiño Crespo-Álvarez,Matthew Stewart,Juan Manuel Reina-Muñoz,Prateek Nagras,Gaurav Vikhe,Mohammad Bakhshalipour,Martin Pinzger,Stefan Rass,Smruti Panigrahi,Giulio Corradi,Niladri Roy,Phillip B. Gibbons,Sabrina M. Neuman,Brian Plancher,Vijay Janapa Reddi,Víctor Mayoral-Vilches,Jason Jabbour,Yu-Shun Hsiao,Zishen Wan,Martiño Crespo-Álvarez,Matthew Stewart,Juan Manuel Reina-Muñoz,Prateek Nagras,Gaurav Vikhe,Mohammad Bakhshalipour,Martin Pinzger,Stefan Rass,Smruti Panigrahi,Giulio Corradi,Niladri Roy,Phillip B. Gibbons,Sabrina M. Neuman,Brian Plancher,Vijay Janapa Reddi

We introduce RobotPerf, a vendor-agnostic bench-marking suite designed to evaluate robotics computing performance across a diverse range of hardware platforms using ROS 2 as its common baseline. The suite encompasses ROS 2 packages covering the full robotics pipeline and integrates two distinct benchmarking approaches: black-box testing, which measures performance by eliminating upper layers and r...