- This paper presents a methodology for detailed evaluation of autonomous automated guided vehicles systems (AGVS) used for material handling. The methodology includes: stand-alone sub-module evaluation, including comprehensive simulations and statistical analysis of the system's sub-modules, along with hardware validation; quantitative system evaluation for integrated system performance investigation; and structured qualitative analyses for identifying strengths and weaknesses not readily apparent. The defined performance measures include aspects from both multi-robot and AGV fields. The developed methodology provides a systematic way to model, experiment with, analyze, and compare different AGVS control methods. To demonstrate the methodology, it was applied to evaluate a recently developed decentralized AGVS control method.